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ACQUISITION  AND 
TECHNOLOGY 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 
3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-3000 


RISK  MANAGEMENT  GUIDE 


Acquisition  reform  has  changed  the  way  the  Department  of  Defense  (DoD)  designs, 
develops,  manufactures,  and  supports  systems.  Our  technical,  business,  and  manage¬ 
ment  approach  for  acquiring  and  operating  systems  has,  and  continues  to,  evolve.  For 
example,  we  no  longer  can  rely  on  military  specifications  and  standards  to  define  and 
control  how  our  developers  design,  build,  and  support  our  new  systems.  Today  we  use 
commercial  hardware  and  software,  promote  open  systems  architecture,  and  encourage 
streamlining  processes,  just  to  name  a  few  of  the  initiatives  that  affect  the  way  we  do 
business.  At  the  same  time,  the  Office  of  the  Secretary  of  Defense  (OSD)  has  reduced  the 
level  of  oversight  and  review  of  programs  and  manufacturers'  plants. 

While  the  new  acquisition  model  gives  government  Program  Managers  and  their  con¬ 
tractors  broader  control  and  more  options  than  they  have  enjoyed  in  the  past,  it  also  ex¬ 
poses  them  to  new  risks.  OSD  recognizes  that  risk  is  inherent  in  any  acquisition  program 
and  considers  it  essential  that  Program  Managers  take  appropriate  steps  to  manage  and 
control  risks. 

In  late  1996,  the  Under  Secretary  of  Defense,  Acquisition  and  Technology  [USD(A&T)j 
tasked  the  Director,  Test,  Systems  Engineering,  and  Evaluation  (DTSE&E)  to  review  DoD 
risk  management  practices  and  techniques.  In  response,  DTSE&E /Systems  Engineering 
established  a  Risk  Management  Working  Group  that  examined  the  Services,  individual 
acquisition  programs,  and  commercial  industry's  treatment  of  risk.  The  results  of  the 
study  served  as  the  basis  for  the  risk  management  section  (2.5.2)  in  the  Defense  Acquisition 
Deskbook.  The  study  also  identified  the  need  to  update  existing  risk  training  material  to 
reflect  the  new  way  DoD  conducts  business. 
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This  document  is  a  product  of  a  joint  effort  among  the  DTSE&E,  the  Defense  Acquisition 
University,  and  the  Defense  Systems  Management  College.  It  is  based  on  the  material 
developed  by  the  DoD  Risk  Management  Working  Group,  included  in  the  Defense 
Acquisition  Deskbook. 


Deputy  Director,  Test,  Systems  President 

Engineering  and  Evaluation/  Defense  Acquisition  University 

Systems  Engineering 


RADM,  SC,  USN 
Commandant 
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PREFACE 


In  December  1995,  the  Under  Secretary  of  Defense,  Acquisition  and  Technology  [USD(A&T)], 
issued  a  memorandum  entitled  Reducing  Life  Cycle  Costs  for  New  and  Fielded  Systems,  in  which 
he  established  the  policy  and  strategy  to  develop  and  field  affordable  weapon  systems  that 
are  responsive  to  user's  needs.  One  of  the  foundations  of  the  strategy  is  the  concept  of  "Cost 
as  An  Independent  Variable"  (CAIV),  the  Department  of  Defense  (DoD)  equivalent  of  com¬ 
mercial  best  practices.  The  CAIV  concept  recognizes  that  "there  are  risks  to  be  taken  and  risks 
to  be  avoided.  When  risks  are  taken,  we  will  put  in  place  appropriate  risk  management  and 
contingency  plans." 

Other  initiatives,  such  as  acquisition  streamlining  and  revision  of  the  DoD  5000  series 
documents,  were  ongoing  when  the  USD(A&T)  memorandum  was  published;  each  af¬ 
fected  program  risk.  Also  at  this  time,  the  DoD  Inspector  General  was  writing  a  critical 
report  of  the  Department's  management  of  risk;  the  report  recommended  measures  to 
control  risk  of  acquisition  programs.  Figure  P-1  shows  some  of  the  initiatives  that  impact 
risk  management. 


Emphasis  on 
Risk 

Management 


ACQUISITION  INITIATIVES 


OPEN  SYSTEMS 


COSTAS  AN  INDEPENDENT  VARIABLE  (CAIV) 


ACQUISITION  STREAMLINING 


COMMERCIAL  ITEMS,  HARDWARE  AND  SOFTWARE 


MILITARY  SPECIFICATIONS  AND  STANDARDS 


REDUCED  PLANT  OVERSIGHT 


SINGLE  PROCESS  INITIATIVE 


REDUCTION  IN  FUNDING 


Figure  P-1.  DoD  Renewed  Emphasis  on  Risk  Management 


With  these  initiatives  as  the  basis,  the  USD(  A&T)  tasked  the  Director,  Test,  Systems  Engi¬ 
neering,  and  Evaluation  (DTSE&E)  to:  (1)  review  DoD  risk  management  practices  and 
techniques,  (2)  determine  whether  new  approaches  were  needed  to  improve  risk  man¬ 
agement,  and  (3)  report  the  results  to  USD(  A&T). 

In  response,  DTSE&E  established  a  Risk  Management  Working  Group  composed  of  mem¬ 
bers  of  the  Office  of  the  Secretary  of  Defense  (OSD)  staff,  representatives  of  the  Services, 
and  members  of  other  DoD  agencies  involved  in  systems  acquisition.  This  group  reviewed 
pertinent  DoD  directives  (DoDD)  and  regulations,  examined  how  the  Services  managed 
risk,  studied  various  examples  of  risk  management  by  companies  in  commercial  indus¬ 
try,  and  looked  at  DoD  training  and  education  activity  in  risk  management.  The  Working 
Group  coordinated  with  other  related  efforts  in  DoD.  For  example,  the  Joint  Aeronautical 
Commanders  Group  Risk  Guide  was  a  valuable  source  of  information.  The  workshops  for 
the  CAIV  Flagship  programs  provided  current,  real-world  examples  of  Program  Manag¬ 
ers  implementing  the  CAIV  initiative  and  risk  management.  Membership  of  the  Working 
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Group  included  a  representative  from  USD(A&T)  Acquisition  Program  Integration/Pro¬ 
gram  Management  (API/PM)  who  kept  members  informed  on  the  status  of  the  Integrated 
Program  Management  Initiative.  Other  sources  of  information  were  the  Software  Engi¬ 
neering  Institute  Risk  Initiative,  the  Open  Systems  Initiative,  and  Safety  and  Cost  Esti¬ 
mating  communities.  DTSE&E  summarized  the  findings  of  the  investigation  and  pre¬ 
sented  the  results  to  the  USD(A&T)  in  July  1996. 


The  findings  and  recommendations  of  the  Working  Group  are  summarized  below. 


•  Focus  of  efforts  is  on  getting  a  product  to  market  at  a  competitive  cost. 

•  Companies  have  either  a  structured  or  informal  Risk  Management  process. 

•  Evolutionary  approaches  help  avoid  or  minimize  risk. 

•  Most  approaches  employ  risk  avoidance,  early  planning,  continuous  assessment,  and  problem¬ 
solving  techniques. 

•  Structured  approaches,  when  they  exist,  are  similar  to  DoD’s  approach  to  Risk  Management. 

The  Working  Group  concluded  that  industry  has  no  magic  formula  for  Risk  Management. 

The  Services 

•  The  Services  differ  in  their  approaches  to  Risk  Management. 

•  Each  approach  has  its  strengths  but  no  one  approach  is  comprehensive. 

•  Consolidation  of  the  strengths  of  each  approach  could  foster  better  Risk  Management  in  DoD. 

The  Working  Group  recommended  that  the  Defense  Acquisition  Deskbook  contain  a  set  of  guidelines 
for  sound  risk  management  practices,  and  further,  that  it  contain  a  set  of  risk  management  definitions 
that  are  comprehensive  and  useful  by  all  the  Components. 

DoD  Policy  _  :  : 

•  The  risk  management  policy  contained  in  DoDD  5000.1  is  not  comprehensive. 

The  Working  Group  recommended  that  DoDD  5000.1  be  amended  to  include  a  more  comprehensive 
set  of  risk  management  policies  that  focuses  on: 

•  The  relationship  between  the  CAIV  concept  and  Risk  Management. 

•  Requirement  that  risk  management  be  prospective  (forward  looking). 

•  Establishment  of  risk  management  as  a  primary  management  technique  to  be  used  by  Program 

Managers  (PMs).  ■  . 
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DoD  Procedures 

•  Risk  Management  procedures  in  DoD  5000.2-R  are  inadequate  to  fully  implement  the  risk  manage¬ 
ment  policy  contained  in  DoDD  5000.1 . 

Procedures  are  lacking  regarding  the: 

-  Scope  of  Risk  Management 

-  Purpose  of  Risk  Management 

-  Role  of  Milestone  Decision  Authorities 

-  Risk  Management’s  support  of  CAIV 

-  Risk  assessment  during  Phase  0. 

•  Some  key  procedures  may  have  been  lost  in  transition  from  DoD  5000.2M  to  DoD  5000.2-R. 

The  Working  Group  recommended  that  procedures  in  DoD  5000.2-R  be  expanded,  using  the  Defense 
Acquisition  Deskbook as  the  expansion  means,  in  order  to  provide  comprehensive  guidance  for  the 
implementation  of  risk  management  policy. 

DoD  Risk  Management  Training _ __ 

•  Risk  management  training  for  the  DoD  acquisition  corps  needs  to  be  updated  and  expanded,  and 
Integrated  Product  Team  (IPT)  and  Overarching  IPT  (OIPT)  personnel  need  to  be  educated  on  the 
new  and  expanding  role  of  risk  management  in  DoD  systems  acquisition. 

•  Risk  Management  knowledge  level  needs  improvement. 

•  Education  is  a  key  to  getting  the  support  of  OIPTs  and  PMs. 

The  Working  Group  recommended  that  the  Defense  Acquisition  University  (DAU)  include  training  for 
Risk  Management  in  all  functional  courses  and  develop  a  dedicated  risk  management  course  for 
acquisition  corps  personnel.  _ _ _ 

DTSE&E  briefed  the  results  to  the  Defense  Manufacturing  Council,  an  advisory  body  to  the 
USD(A&T),  which  directed  that  the  recommendations  be  incorporated  in  the  Defense  Acquisi¬ 
tion  Deskbook.  Following  that  guidance,  DTSE&E  wrote  the  risk  management  portions  of  the 
Deskbook. 

The  Risk  Deskbook  write-up  forms  the  basis  for  this  Guide.  The  goal  of  the  Risk  Management 
Guide  is  to  provide  acquisition  professionals  and  program  management  offices  with  a 
reference  for  dealing  with  system  acquisition  risks.  It  has  been  designed  as  an  aid  in 
classroom  instruction  and  as  a  reference  for  practical  applications. 

This  Guide  reflects  the  efforts  of  many  people.  Mr.  Mark  Schaeffer,  Deputy  Director,  Sys¬ 
tems  Engineering,  DTSE&E,  who  chaired  the  Risk  Management  Working  Group  and  Mr. 
Mike  Zsak  and  Mr.  Tom  Parry  from  the  DTSE&E,  Systems  Engineering  Support  Office, 
were  the  driving  force  behind  the  risk  management  initiative.  Mr.  Paul  McMahon  and 
Mr.  Bill  Bahnmaier  from  the  DSMC  faculty  and  Mr.  Greg  Caruth,  Ms.  Debbie  Gonzalez, 
SFC  Frances  Battle,  USA,  SSgt  Gerald  Gilchrist,  Sr.,  USAF,  from  the  DSMC  Press  guided 
the  composition  of  the  Guide.  Special  recognition  goes  to  the  Institute  for  Defense  Analy¬ 
ses  team  composed  of  Mr.  Louis  Simpleman,  Mr.  Ken  Evans,  Mr.  Jim  Lloyd,  and  Mr. 
Gerald  Pike,  who  compiled  the  data  and  wrote  major  portions  of  the  text. 
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INTRODUCTION 


Risk  has  always  been  a  concern  in  the  ac¬ 
quisition  of  Department  of  Defense  (DoD) 
systems.  The  acquisition  process  itself  is 
designed,  to  a  large  degree,  to  allow  risks 
to  be  controlled  from  conception  to  deliv¬ 
ery  of  a  system.  Unfortunately,  in  the  past, 
some  Program  Managers  (PMs)  and  deci¬ 
sion  makers  have  viewed  risk  as  some¬ 
thing  to  be  avoided.  Any  program  that  had 
risk  was  subject  to  intense  review  and 
oversight.  This  attitude  has  changed.  DoD 
managers  recognize  that  risk  is  inherent 
in  any  program  and  that  it  is  necessary  to 
analyze  future  program  events  to  identify 
potential  risks  and  take  measures  to 
handle  them. 

Risk  management  is  concerned  with  the 
outcome  of  future  events,  whose  exact  out¬ 
come  is  unknown,  and  with  how  to  deal 
with  these  uncertainties,  i.e.,  a  range  of 
possible  outcomes.  In  general,  outcomes 
are  categorized  as  favorable  or  unfavor¬ 
able,  and  risk  management  is  the  art  and 
science  of  planning,  assessing,  and  han¬ 
dling  future  events  to  ensure  favorable  out¬ 
comes.  The  alternative  to  risk  management 
is  crisis  management,  a  resource-intensive 
process  that  is  normally  constrained  by  a 
restricted  set  of  available  options. 

1.1  PURPOSE  AND  SCOPE 

This  Risk  Management  Guide  is  designed  to 
provide  acquisition  professionals  and  pro¬ 
gram  management  offices  (PMOs)  with  a 
reference  book  for  dealing  with  system 


acquisition  risks.  It  is  intended  to  be  useful 
as  an  aid  in  classroom  instruction  and  as  a 
reference  book  for  practical  applications. 
Most  of  the  material  in  this  Guide  is  derived 
from  the  Defense  Acquisition  Deskbook.  Read¬ 
ers  should  refer  to  Paragraph  2.5.2  of  the 
Deskbook  for  any  new  information. 

1.2  ORGANIZATION  OF  THE  GUIDE 

The  Risk  Management  Guide  discusses  risk 
and  risk  management,  defines  terms,  and 
introduces  basic  risk  management  concepts 
(Chapter  2). 

Chapter  3  examines  risk  management  con¬ 
cepts  relative  to  the  DoD  acquisition  pro¬ 
cess.  It  illustrates  how  risk  management  is 
an  integral  part  of  program  management, 
describes  interaction  with  other  acquisition 
processes,  and  identifies  and  discusses  the 
various  types  of  acquisition  risks. 

Chapter  4  discusses  the  implementation  of 
a  risk  management  program  from  the  per¬ 
spective  of  a  PMO.  This  chapter  focuses  on 
practical  application  issues  such  as  risk 
management  program  design  options, 
PMO  risk  management  organizations,  and 
criteria  for  a  risk  management  information 
system  (MIS). 

Chapter  5,  the  final  chapter,  describes  a 
number  of  techniques  that  address  the  as¬ 
pects  (phases)  of  risk  management,  i.e., 
planning,  assessment,  handling,  and 
monitoring. 
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This  Guide  is  a  source  of  background  infor¬ 
mation  and  provides  a  starting  point  for  a 
risk  management  program.  None  of  the 
material  is  mandatory,  PMs  should  tailor 
the  approaches  and  techniques  to  fit  their 
programs. 

The  Risk  Management  Guide  also  contains 
appendices  that  are  intended  to  serve  as 
reference  material  and  provide  backup  de¬ 
tail  for  some  of  the  concepts  presented  in 
the  main  portion  of  the  Guide. 

1.3  APPROACH  TO  RISK 
MANAGEMENT 

Based  on  the  DoD  model  contained  in  the 
Deskbook  (described  in  Chapter  2),  this 
Guide  emphasizes  a  risk  management  ap¬ 
proach  that  is  disciplined,  forward  look¬ 
ing,  and  continuous. 

In  1986,  the  Government  Accounting  Of¬ 
fice  (GAO),  as  part  of  an  evaluation  of  DoD 
policies  and  procedures  for  technical  risk 
assessments,  developed  a  set  of  criteria  as 
an  approach  to  good  risk  assessments. 
These  criteria,  with  slight  modification, 
apply  to  all  aspects  of  risk  management 
and  are  encompassed  in  the  Guide's  ap¬ 
proach.  They  are: 

(1)  Planned  Procedures.  Risk  manage¬ 
ment  is  planned  and  systematic. 

(2)  Prospective  Assessment.  Potential 
future  problems  are  considered,  not  just 
current  problems. 

(3)  Attention  to  Technical  Risk.  There 
is  explicit  attention  to  technical  risk. 

(4)  Documentation.  All  aspects  of  the 
risk  management  program  are  recorded 
and  data  maintained. 

(5)  Continual  Process.  Risk  assess¬ 
ments  are  made  throughout  the  acquisition 


process;  handling  activities  are  continually 
evaluated  and  changed  if  necessary;  and 
critical  risk  areas  are  always  monitored. 

While  these  criteria  are  not  solely  sufficient 
to  determine  the  "health"  of  a  program, 
they  are  important  indicators  of  how  well 
a  risk  management  process  is  being 
implemented. 

1.4  DOD  RISK  MANAGEMENT 
POLICIES  AND  PROCEDURES 

DoD  policies  and  procedures  that  address 
risk  management  for  acquisition  programs 
are  contained  in  four  key  DoD  documents. 
DoDD  5000.1  contains  the  policy  on  risk 
management  and  is  amplified  further  by 
the  information  in  DoD  5000.2-R.  The  lat¬ 
ter  document  integrates  risk  management 
into  the  acquisition  process,  describes  the 
relationship  between  risk  and  various  ac¬ 
quisition  functions,  and  establishes  some 
reporting  requirements.  DoDD  5000.4  and 
DoD  5000.4-M  address  risk  and  cost  analy¬ 
sis  guidance  as  they  apply  to  the  Office  of 
the  Secretary  of  Defense.  Appendix  A  is  an 
extract  of  existing  risk  management  poli¬ 
cies  and  procedures  from  all  of  these 
documents. 

The  DoD  5000  series  contains  strong  state¬ 
ments  on  risk  management  but  requires 
elaboration  to  help  the  PM  establish  an  ef¬ 
fective  risk  management  program.  The  in¬ 
formation  furnished  in  the  Risk  Manage¬ 
ment  section  of  the  Deskbook  supports  and 
expands  the  contents  of  the  DoD  5000 
series. 

The  DoD  risk  management  policies  and 
procedures  provide  the  basis  for  this  Guide, 
which  complements  the  Deskbook  by  elabo¬ 
rating  on  risk  management  concepts  and 
by  providing  greater  detail  for  applying 
techniques. 
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RISK  AND  RISK  MANAGEMENT 


2.1  INTRODUCTION 

This  Chapter  introduces  the  concepts  of 
risk  and  risk  management  by  explaining 
the  DoD  risk-related  definitions  and  by 
identifying  the  characteristics  of  acquisi¬ 
tion  risks.  It  also  presents  and  discusses  a 
structured  concept  for  risk  management 
and  its  five  subordinate  processes. 

2.2  OVERVIEW 

The  DoD  risk  management  concept  is 
based  on  the  principles  that  risk  manage¬ 
ment  must  be  forward-looking,  structured, 
informative,  and  continuous.  The  key  to 
successful  risk  management  is  early  plan¬ 
ning  and  aggressive  execution.  Good  plan¬ 
ning  enables  an  organized,  comprehensive, 
and  iterative  approach  for  identifying  and 
assessing  the  risk  and  handling  options 
necessary  to  refine  a  program  acquisition 
strategy.  To  support  these  efforts,  assess¬ 
ments  should  be  performed  as  early  as  pos¬ 
sible  in  the  life  cycle  to  ensure  that  critical 
technical,  schedule,  and  cost  risks  are  ad¬ 
dressed  with  mitigation  actions  incorpo¬ 
rated  into  program  planning  and  budget 
projections. 

PMs  should  update  program  risk  assess¬ 
ments  and  tailor  their  management  strat¬ 
egies  accordingly.  Early  information 
gives  them  data  that  helps  when  writing 
a  Request  for  Proposal  and  assists  in 
Source  Selection  planning.  As  a  program 
progresses,  new  information  improves 


insight  into  risk  areas,  thereby  allowing 
the  development  of  effective  handling 
strategies.  The  net  result  promotes  ex¬ 
ecutable  programs. 

Effective  risk  management  requires  in¬ 
volvement  of  the  entire  program  team  and 
also  requires  help  from  outside  experts 
knowledgeable  in  critical  risk  areas  (e.g., 
threat,  technology,  design,  manufacturing, 
logistics,  schedule,  and  cost).  In  addition, 
the  risk  management  process  should  cover 
hardware,  software,  the  human  element, 
and  integration  issues.  Outside  experts 
may  include  representatives  from  the  user, 
laboratories,  contract  management,  test, 
logistics,  and  sustainment  communities, 
and  industry.  Users,  essential  participants 
in  program  trade  analyses,  should  be  part 
of  the  assessment  process  so  that  an  ac¬ 
ceptable  balance  among  cost,  schedule, 
performance,  and  risk  can  be  reached.  A 
close  relationship  between  the  Govern¬ 
ment  and  industry,  and  later  with  the  se¬ 
lected  contractor(s),  promotes  an  under¬ 
standing  of  program  risks  and  assists  in 
developing  and  executing  the  manage¬ 
ment  efforts. 

Successful  risk  management  programs 
generally  have  the  following  characteristics: 

•  Feasible,  stable,  and  well-understood 
user  requirements  and  threat; 

•  A  close  relationship  with  user,  indus¬ 
try,  and  other  appropriate  participants; 
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•  A  planned  and  structured  risk  man¬ 
agement  process,  integral  to  the  acquisition 
process; 

•  An  acquisition  strategy  consistent 
with  risk  level  and  risk-handling  strategies; 

•  Continual  reassessment  of  program 
and  associated  risks; 

•  A  defined  set  of  success  criteria  for  all 
cost,  schedule,  and  performance  elements, 
e.g..  Acquisition  Program  Baseline  (APB) 
thresholds; 

•  Metrics  to  monitor  effectiveness  of 
risk-handling  strategies; 

•  Effective  Test  and  Evaluation  Program; 

•  Formal  documentation. 

PMs  should  follow  the  guidelines  below 
to  ensure  that  a  management  program 
possesses  the  above  characteristics. 

•  Assess  program  risks,  using  a  struc¬ 
tured  process,  and  develop  strategies  to 
manage  these  risks  throughout  each  acqui¬ 
sition  phase. 

•  Identify  early  and  intensively  manage 
those  design  parameters  that  critically  af¬ 
fect  cost,  capability,  or  readiness. 

•  Use  technology  demonstrations/ 
modeling/simulation  and  aggressive 
prototyping  to  reduce  risks. 

•  Use  test  and  evaluation  as  a  means  of 
quantifying  the  results  of  the  risk-handling 
process. 

•  Include  industry  and  user  participa¬ 
tion  in  risk  management. 

•  Use  Developmental  Test  and  Evalua¬ 
tion  (DT&E)  and  early  operational  assess¬ 
ments  when  appropriate. 


•  Establish  a  series  of  "risk  assessment 
reviews"  to  evaluate  the  effectiveness  of 
risk  handling  against  clearly  defined  suc¬ 
cess  criteria. 

•  Establish  the  means  and  format  to 
communicate  risk  information  and  to  train 
participants  in  risk  management. 

•  Prepare  an  assessment  training  pack¬ 
age  for  members  of  the  program  office  and 
others,  as  needed. 

•  Acquire  approval  of  accepted  risks  at 
the  appropriate  decision  level. 

In  general,  management  of  software  risk 
is  the  same  as  management  of  other  types 
of  risk  and  techniques  that  apply  to  hard¬ 
ware  programs  are  equally  applicable  to 
software  intensive  programs.  However, 
some  characteristics  of  software  make  this 
type  of  risk  management  different,  prima¬ 
rily  because  it  is  difficult  to: 

•  Identify  software  risk. 

•  Estimate  the  time  and  resources  re¬ 
quired  to  develop  new  software,  resulting 
in  potential  risks  in  cost  and  schedule. 

•  Test  software  completely  because  of 
the  number  of  paths  that  can  be  followed 
in  the  logic  of  the  software. 

•  Develop  new  programs  because  of  the 
rapid  changes  in  information  technology 
and  an  ever-increasing  demand  for  qual¬ 
ity  software  personnel. 

2.3  RISK  MANAGEMENT 

STRUCTURE  AND  DEFINITIONS 

Although  each  risk  management  strategy 
depends  upon  the  nature  of  the  system  be¬ 
ing  developed,  research  reveals  that  good 
strategies  contain  the  same  basic  processes 
and  structure  shown  in  Figure  2-1.  This 
structure  is  sometimes  also  referred  to  as 
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Figure  2-1.  Risk  Management  Structure 


the  Risk  Management  Process  Model.  The 
application  of  these  processes  vary  with  ac¬ 
quisition  phases  and  the  degree  of  system 
definition;  all  should  be  integrated  into  the 
program  management  function. 

The  elements  of  the  structure  are  discussed 
in  the  following  paragraphs  of  this  Chap¬ 
ter;  however,  in  order  to  form  a  basis  for 
discussion,  the  Deskbook  definitions  for  the 
processes  and  elements  of  risk  manage¬ 
ment  include: 

Risk  is  a  measure  of  the  potential  inabil¬ 
ity  to  achieve  overall  program  objectives 
within  defined  cost,  schedule/and  techni¬ 
cal  constraints  and  has  two  components: 
(1)  the  probability  (or  likelihood)  of  failing 
to  achieve  a  particular  outcome,  and  (2)  the 
consequences  (or  impact)  of  failing  to 
achieve  that  outcome. 

Risk  management  is  the  act  or  practice 
of  dealing  with  risk.  It  includes  planning 
for  risk,  assessing  (identifying  and  analyz¬ 
ing)  risk  areas,  developing  risk-handling 
options,  monitoring  risks  to  determine  how 
risks  have  changed,  and  documenting  the 
overall  risk  management  program. 

Risk  planning  is  the  process  of  devel¬ 
oping  and  documenting  an  organized, 
comprehensive,  and  interactive  strategy 
and  methods  for  identifying  and  tracking 


risk  areas,  developing  risk-handling  plans, 
performing  continuous  risk  assessments  to 
determine  how  risks  have  changed,  and  as¬ 
signing  adequate  resources. 

Risk  events,  things  that  could  go  wrong 
for  a  program  or  system,  are  elements  of 
an  acquisition  program  that  should  be  as¬ 
sessed  to  determine  the  level  of  risk.  The 
events  should  be  defined  to  a  level  that  an 
individual  can  comprehend  the  potential 
impact  and  its  causes.  For  example,  a  po¬ 
tential  risk  event  for  a  turbine  engine  could 
be  turbine  blade  vibration.  There  could  be 
a  series  of  potential  risk  events  that  should 
be  selected,  examined,  and  assessed  by 
subject-matter  experts. 

The  relationship  between  the  two  compo¬ 
nents  of  risk — probability  and  conse¬ 
quence — is  complex.  To  avoid  obscuring 
the  results  of  an  assessment,  the  risk  asso¬ 
ciated  with  an  event  should  be  character¬ 
ized  in  terms  of  its  two  components.  There 
is  still  a  need  for  backup  documentation 
containing  the  supporting  data  and  assess¬ 
ment  rationale. 

Risk  assessment  is  the  process  of  iden¬ 
tifying  and  analyzing  program  areas  and 
critical  technical  process  risks  to  increase 
the  likelihood  of  meeting  cost,  schedule, 
and  performance  objectives.  Risk  identifica¬ 
tion  is  the  process  of  examining  the 
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program  areas  and  each  critical  technical 
process  to  identify  and  document  the  as¬ 
sociated  risk.  Risk  analysis  is  the  process  of 
examining  each  identified  risk  area  or  pro¬ 
cess  to  refine  the  description  of  the  risk,  iso¬ 
lating  the  cause,  and  determining  the  ef¬ 
fects.  It  includes  risk  rating  and 
prioritization  in  which  risk  events  are  de¬ 
fined  in  terms  of  their  probability  of  oc¬ 
currence,  severity  of  consequence  (or  im¬ 
pacts),  and  relationship  to  other  risk  areas 
or  processes. 

Risk  handling  is  the  process  that  identi¬ 
fies,  evaluates,  selects,  and  implements  op¬ 
tions  in  order  to  set  risk  at  acceptable  levels 
given  program  constraints  and  objectives. 
This  includes  the  specifics  on  what  should 
be  done,  when  it  should  be  accomplished, 
who  is  responsible,  and  associated  cost  and 
schedule.  The  most  appropriate  strategy  is 
selected  from  these  handling  options.  For 
purposes  of  the  Guide,  risk  handling  is  an 
all-encompassing  term  whereas  risk  mitiga¬ 
tion  is  one  subset  of  risk  handling. 

Risk  monitoring  is  the  process  that  sys¬ 
tematically  tracks  and  evaluates  the  per¬ 
formance  of  risk-handling  actions  against 
established  metrics  throughout  the  acqui¬ 
sition  process  and  develops  further  risk¬ 
handling  options,  as  appropriate. 

Risk  documentation  is  recording,  main¬ 
taining,  and  reporting  assessments,  han¬ 
dling  analysis  and  plans,  and  monitoring 
results.  It  includes  all  plans,  reports  for  the 
PM  and  decision  authorities,  and  report¬ 
ing  forms  that  may  be  internal  to  the  PMO. 

2.4  RISK  DISCUSSION 

Implicit  in  the  definition  of  risk  is  the  con¬ 
cept  that  risks  are  future  events  and  that 
there  is  uncertainty  associated  with  the  pro¬ 
gram  if  these  events  occur.  Therefore,  there 
is  a  need  to  determine,  as  much  as  possible, 
the  probability  of  a  risk  event  occurring  and 


to  estimate  the  impact  (consequences)  if  it 
occurs.  The  combination  of  these  two  fac¬ 
tors  determines  severity.  For  example,  an 
event  with  a  low  probability  of  occurring, 
yet  with  severe  consequences,  may  be  a 
candidate  for  handling.  Conversely,  an 
event  with  a  high  probability  of  happening, 
but  the  consequences  of  which  do  not  af¬ 
fect  a  program,  may  be  acceptable  and  re¬ 
quire  no  handling. 

To  reduce  uncertainty  and  apply  the  defini¬ 
tion  of  risk  to  acquisition  programs,  PMs 
must  be  familiar  with  the  types  of  acquisi¬ 
tion  risks,  understand  risk  terminology,  and 
know  how  to  measure  risk.  These  topics  are 
addressed  in  the  next  several  sections. 

2.4.1  Characteristics  of 
Acquisition  Risk 

Acquisition  programs  tend  to  have  numer¬ 
ous,  often  interrelated,  risks.  They  are  not 
always  obvious;  relationships  may  be  ob¬ 
scure;  and  they  may  exist  at  all  program 
levels  throughout  the  life  of  a  program. 
Risks  are  in  the  PMO  (program  plans,  etc.); 
in  support  provided  by  other  Government 
agencies;  in  threat  assessment;  and  in 
prime  contractor  processes,  engineering 
and  manufacturing  processes,  and  tech¬ 
nology.  The  interrelationship  among  risk 
events  may  cause  an  increase  in  one  be¬ 
cause  of  the  occurrence  of  another.  For 
example,  a  slip  in  schedule  for  an  early 
test  event  may  adversely  impact  subse¬ 
quent  tests,  assuming  a  fixed  period  of  test 
time  is  available. 

Another  important  risk  characteristic  is  the 
time  period  before  a  risk  future  event  oc¬ 
curs;  because  time  is  critical  in  determin¬ 
ing  risk-handling  options.  If  an  event  is 
imminent,  the  PMO  must  resort  to  crisis 
management.  An  event  that  is  far  enough 
in  the  future  to  allow  management  actions 
may  be  controllable.  The  goal  is  to  avoid 
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the  need  to  revert  to  problem  solving  by 
managing  risk. 

An  event's  probability  of  occurrence  and 
consequences  may  change  as  the  develop¬ 
ment  process  proceeds  and  information 
becomes  available.  Therefore,  throughout 
the  development  phase,  PMOs  should  re¬ 
evaluate  known  risks  on  a  periodic  basis 
and  examine  the  program  for  new  risks. 

2.4.2  Program  Products,  Processes, 

Risk  Areas,  and  Risk  Events 

Program  risk  includes  all  risk  events  and 
their  relationships  to  each  other.  It  is  a  top- 
level  assessment  of  impact  to  the  program 
when  all  risk  events  at  the  lower  levels  of 
the  program  are  considered.  Program  risk 
may  be  a  roll-up  of  all  low-level  events; 
however,  most  likely,  it  is  a  subjective 
evaluation  of  the  known  risks  by  the  PMO, 
based  on  the  judgment  and  experience  of 
experts.  Any  roll-up  of  program  risks 
must  be  carefully  done  to  prevent  key  risk 
issues  from  "slipping  through  the  cracks." 
Identifying  program  risk  is  worthwhile 
because  it  forces  the  PMO  to  consider  re¬ 
lationships  among  all  risks  and  may  iden¬ 
tify  potential  areas  of  concern  that  would 
have  otherwise  been  overlooked.  One  of 
the  greatest  strengths  of  a  formal,  continu¬ 
ous  risk  management  process  is  the  pro¬ 
active  quest  to  identify  risk  events  for  han¬ 
dling  and  the  reduction  of  uncertainty  that 
results  from  handling  actions. 

A  program  office  has  continuous  demands 
on  its  time  and  resources.  It  is,  at  best, 
difficult,  and  probably  impossible,  to  assess 
every  potential  area  and  process.  To  manage 
risk,  the  PMOs  should  focus  on  the  critical 
areas  that  could  affect  the  outcome  of  their 
programs.  Work  Breakdown  Structure  (WBS) 
product  and  process  elements  and  industrial 
engineering  and  manufacturing  processes 
contain  most  of  the  significant  risk  events. 


Risk  events  are  determined  by  examining 
each  WBS  element  and  process  in  terms  of 
sources  or  areas  of  risk.  Broadly  speaking, 
these  sources  generally  can  be  grouped  as 
cost,  schedule,  and  performance,  with  the 
latter  including  technical  risk.  Following  are 
some  typical  risk  areas: 

•  Threat.  The  sensitivity  of  the  program 
to  uncertainty  in  the  threat  description,  the 
degree  to  which  the  system  design  would 
have  to  change  if  the  threat's  parameters 
change,  or  the  vulnerability  of  the  program 
to  foreign  intelligence  collection  efforts 
(sensitivity  to  threat  countermeasure). 

•  Requirements.  The  sensitivity  of  the 
program  to  uncertainty  in  the  system  de¬ 
scription  and  requirements  except  for  those 
caused  by  threat  uncertainty. 

•  Design.  The  ability  of  the  system  con¬ 
figuration  to  achieve  the  program's  engi¬ 
neering  objectives  based  on  the  available 
technology,  design  tools,  design  maturity, 
etc. 

•  Test  and  Evaluation  (T&E).  The  ad¬ 
equacy  and  capability  of  the  T&E  program 
to  assess  attainment  of  significant  perfor¬ 
mance  specifications  and  determine 
whether  the  systems  are  operationally  ef¬ 
fective  and  suitable. 

•  Modeling  and  Simulation  (M&S). 

The  adequacy  and  capability  of  M&S  to 
support  all  phases  of  a  program  using  veri¬ 
fied,  valid,  and  accredited  M&S. 

•  Technology.  The  degree  to  which  the 
technology  proposed  for  the  program  has 
been  demonstrated  as  capable  of  meeting 
all  of  the  program's  objectives. 

•  Logistics.  The  ability  of  the  system 
configuration  to  achieve  the  program's 
logistics  objectives  based  on  the  system 
design,  maintenance  concept,  support 
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system  design,  and  availability  of  support 
resources. 

•  Production.  The  ability  of  the  system 
configuration  to  achieve  the  program's  pro¬ 
duction  objectives  based  on  the  system  de¬ 
sign,  manufacturing  processes  chosen,  and 
availability  of  manufacturing  resources 
such  as  facilities  and  personnel. 

•  Concurrency.  The  sensitivity  of  the 
program  to  uncertainty  resulting  from  the 
combining  or  overlapping  of  life-cycle 
phases  or  activities. 

•  Capability  of  Developer.  The  ability 
of  the  developer  to  design,  develop,  and 
manufacture  the  system.  The  contractor 
should  have  the  experience,  resources,  and 
knowledge  to  produce  the  system. 

•  Cost/Funding.  The  ability  of  the  sys¬ 
tem  to  achieve  the  program's  life-cycle 
cost  objectives.  This  includes  the  effects  of 
budget  and  affordability  decisions  and  the 
effects  of  inherent  errors  in  the  cost  esti¬ 
mating  technique(s)  used  (given  that  the 
technical  requirements  were  properly 
defined). 

•  Management.  The  degree  in  which 
program  plans  and  strategies  exist  and  are 
realistic  and  consistent.  The  Government's 
acquisition  team  should  be  qualified  and 
sufficiently  staffed  to  manage  the  program. 

•  Schedule.  The  adequacy  of  the  time 
allocated  for  performing  the  defined  tasks, 
e.g.,  developmental,  production,  etc.  This 
factor  includes  the  effects  of  programmatic 
schedule  decisions,  the  inherent  errors  in 
the  schedule  estimating  technique  used, 
and  external  physical  constraints. 

Critical  risk  processes  are  the  developer's 
engineering  and  production  processes 
which,  historically,  have  caused  the  most 
difficulty  during  the  development  and/or 
production  phases  of  acquisition  programs. 


These  processes  include,  but  are  not  lim¬ 
ited  to,  design,  test,  production,  facilities, 
logistics,  and  management.  These  pro¬ 
cesses  are  included  in  the  critical  risk  ar¬ 
eas  and  are  addressed  separately  to  em¬ 
phasize  that  they  focus  on  processes.  DoD 
4245.7-M,  Transition  from  Development  to 
Production,  describes  them  using  tem¬ 
plates.  See  Figure  2-2  for  an  example  of 
the  template  for  product  development. 
The  templates  are  the  result  of  a  Defense 
Science  Board  task  force,  composed  of 
Government  and  industry  experts,  who 
identified  engineering  processes  and  con¬ 
trol  methods  to  minimize  risk  in  both  Gov¬ 
ernment  and  industry.  The  task  force  de¬ 
fined  these  critical  events  in  terms  of  the 
templates,  which  are  briefly  discussed 
later.  The  figure  also  shows  funding  as  a 
process  that,  unlike  others,  is  a  Govern¬ 
ment  process. 

Additional  areas,  such  as  manpower,  en¬ 
vironmental  impact,  systems  safety  and 
health,  and  systems  engineering,  that  are 
analyzed  during  program  plan  develop¬ 
ment  provide  indicators  for  additional  risk. 
The  PMO  should  consider  these  areas  for 
early  assessment  since  failure  to  do  so 
could  cause  dire  consequences  in  the 
program's  latter  phases. 

In  addition,  PMs  should  address  the  uncer¬ 
tainty  associated  with  security — an  area 
sometimes  overlooked  by  developers  but 
addressed  in  the  Acquisition  System 
Protection  (ASP)  section  of  the  Deskbook  and 
Air  Force  Pamphlet  ASPWG  PH-1,  Acqui¬ 
sition  System  Protection  Program  Work  Book, 
September  1994.  However,  in  addition  to 
the  guidance  given  there,  PMs  must  rec¬ 
ognize  that,  in  the  past,  classified  programs 
have  experienced  difficulty  in  access,  fa¬ 
cilities,  clearances,  and  visitor  control.  Fail¬ 
ure  to  manage  these  aspects  of  a  classified 
program  could  adversely  affect  cost  and 
schedule. 
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PRODUCT 


Figure  2-2.  Critical  Process  Areas  and  Templates 


2.5  RISK  PLANNING 

2.5.1  Purpose  of  Risk  Plans 

Risk  planning  is  the  detailed  formulation 
of  a  program  of  action  for  the  management 
of  risk.  It  is  the  process  to: 

•  Develop  and  document  an  organized, 
comprehensive,  and  interactive  risk  man¬ 
agement  strategy. 

•  Determine  the  methods  to  be  used 
to  execute  a  PM's  risk  management 
strategy. 

•  Plan  for  adequate  resources. 

Risk  planning  is  iterative  and  includes  de¬ 
scribing  and  scheduling  the  activities  and 
process  to  assess  (identify  and  analyze), 
handle,  monitor,  and  document  the  risk 
associated  with  a  program.  The  result  is  the 
Risk  Management  Plan  (RMP). 


2.5.2  Risk  Planning  Process 

The  PMO  should  periodically  review  the 
plan  and  revise  it,  if  necessary.  Some 
events  such  as:  (1)  a  change  in  acquisition 
strategy,  (2)  preparation  for  a  major  deci¬ 
sion  point,  (3)  technical  audits  and  re¬ 
views,  (4)  an  update  of  other  program 
plans,  and  (5)  preparation  for  a  Program 
Objective  Memorandum  (POM)  submis¬ 
sion  may  drive  the  need  to  update  an 
existing  plan. 

Planning  begins  by  developing  and  docu¬ 
menting  a  risk  management  strategy.  Early 
efforts  establish  the  purpose  and  objective, 
assign  responsibilities  for  specific  areas, 
identify  additional  technical  expertise 
needed,  describe  the  assessment  process 
and  areas  to  consider,  delineate  procedures 
for  consideration  of  handling  options, 
define  a  risk  rating  scheme,  dictate  the  re¬ 
porting  and  documentation  needs,  and 
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establish  report  requirements  and  monitor¬ 
ing  metrics.  This  planning  should  also  ad¬ 
dress  evaluation  of  the  capabilities  of  po¬ 
tential  sources  as  well  as  early  industry  in¬ 
volvement  and  program. 

The  PM's  strategy  to  manage  risk  provides 
the  program  team  with  direction  and  basis 
for  planning.  Initially  formalized  during  a 
program's  Concept  Exploration  Phase  and 
updated  for  each  subsequent  program 
phase,  the  strategy  should  be  reflected  in 
the  program's  acquisition  strategy,  which 
with  requirement  and  threat  documents, 
known  risks,  and  system  and  program 
characteristics  are  sources  of  information 
for  PMO  use  to  devise  a  strategy  and  be¬ 
gin  developing  a  Risk  Management  Plan. 
Since  the  program's  risks  are  affected  by 
the  Government  and  contractor  team's  abil¬ 
ity  to  develop  and  manufacture  the  system, 
industry  can  provide  valuable  insight  into 
this  area  of  consideration. 

The  plan  is  the  road  map  that  tells  the  Gov¬ 
ernment  and  contractor  team  how  to  get 
from  where  the  program  is  today  to  where 
the  PM  wants  it  to  be  in  the  future.  The 
key  to  writing  a  good  plan  is  to  provide 
the  necessary  information  so  the  program 
team  knows  the  objectives,  goals,  and  the 
PMO's  risk  management  process.  Since  it 
is  a  map,  it  may  be  specific  in  some  areas, 
such  as  the  assignment  of  responsibilities 
for  Government  and  contractor  partici¬ 
pants  and  definitions,  and  general  in  other 
areas  to  allow  users  to  choose  the  most  ef¬ 
ficient  way  to  proceed.  For  example,  a  de¬ 
scription  of  techniques  that  suggests  sev¬ 
eral  methods  for  evaluators  to  use  to  as¬ 
sess  risk  is  appropriate,  since  every  tech¬ 
nique  has  advantages  and  disadvantages 
depending  on  the  situation. 

Appendix  B  contains  two  examples  of  a  risk 
plan  and  a  summary  of  the  format  is  shown 
in  Figure  2-3. 


Introduction 
Program  Summary 
Definitions 
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Risk  Management  Process  and  Procedures 

Risk  Planning 

Risk  Assessment 

Risk  Handling 

Risk  Monitoring 

Risk  Management  Information  System, 
Documentation  and  Reports 


Figure  2-3.  A  Risk  Management  Plan  Format 

In  a  decentralized  PMO  risk  management 
organization,  the  program's  risk  manage¬ 
ment  coordinator  may  be  responsible  for 
risk  management  planning.  See  Sections 
4.4,  Risk  Management  Organizations,  and 
5.3,  Risk  Planning  Techniques. 

2.6  RISK  ASSESSMENT 

2.6.1  Purpose  of  Risk  Assessments 

The  primary  objective  of  assessments  is  to 
identify  and  analyze  program  risks  so  that 
the  most  critical  among  them  may  be  con¬ 
trolled.  Assessments  are  factors  that  man¬ 
agers  should  consider  in  setting  cost,  sched¬ 
ule,  and  performance  objectives  because 
they  provide  an  indication  of  the  likelihood 
of  achieving  the  desired  outcomes. 

2.6.2  Risk  Assessment  Process 

Risk  assessment  is  the  problem  definition 
stage  of  management  that  identifies  and 
analyzes  (quantifies)  program  events  in 
terms  of  probability  and  consequences.  The 
results  form  the  basis  for  most  risk  man¬ 
agement  actions.  It  is  probably  the  most 
difficult  and  time-consuming  part  of  the 
management  process.  There  are  no  quick 
answers  or  shortcuts.  Tools  are  available 
to  assist  evaluators  in  assessing  risk,  but 
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none  are  totally  suitable  for  any  program 
and  may  be  highly  misleading  if  the  user 
does  not  understand  how  to  apply  them 
or  interpret  the  results.  Despite  its  complex¬ 
ity,  risk  assessment  is  one  of  the  most  im¬ 
portant  phases  of  the  risk  process  because 
the  caliber  and  quality  of  assessments  de¬ 
termine  the  effectiveness  of  a  management 
program. 

The  components  of  assessment,  identifi¬ 
cation  and  analysis,  are  performed  se¬ 
quentially  with  identification  being  the 
first  step. 

Risk  identification  begins  by  compiling  the 
program's  risk  events.  PMOs  should  exam¬ 
ine  and  identify  program  events  by  reduc¬ 
ing  them  to  a  level  of  detail  that  permits 
an  evaluator  to  understand  the  significance 
of  any  risk  and  identify  its  causes,  i.e.,  risk 
drivers.  This  is  a  practical  way  of  address¬ 
ing  the  large  and  diverse  number  of 
potential  risks  that  often  occur  in  acquisi¬ 
tion  programs.  For  example,  a  WBS  level  4 
or  5  element  may  generate  several  risk 
events  associated  with  a  specification  or 
function,  e.g.,  failure  to  meet  turbine  blade 
vibration  requirements  for  an  engine  tur¬ 
bine  design. 

Risk  events  are  best  identified  by  examin¬ 
ing  each  WBS  product  and  process  element 
in  terms  of  the  sources  or  areas  of  risk,  as 
previously  described  in  Paragraph  2.4.2. 

Risks  are  those  events  that  evaluators  (after 
examining  scenarios,  WBS,  or  processes) 
determine  would  adversely  affect  the 
program.  Evaluators  may  initially  rank 
events  by  probability  and  consequence  of 
occurrence  before  beginning  analysis  to 
focus  on  those  most  critical. 

Risk  analysis  is  a  technical  and  systematic 
process  to  examine  identified  risks,  isolate 
causes,  determine  the  relationship  to  other 


risks,  and  express  the  impact  in  terms  of 
probability  and  consequences. 

In  practice,  the  distinction  between  risk 
identification  and  risk  analysis  is  often 
blurred  because  there  is  some  risk  analy¬ 
sis  that  occurs  during  the  identification 
process.  For  example,  if,  in  the  process 
of  interviewing  an  expert,  a  risk  is  iden¬ 
tified,  it  is  logical  to  pursue  information 
on  the  probability  of  it  occurring,  the  con¬ 
sequences,  the  time  associated  with  the 
risk  (i.e.,  when  it  might  occur),  and  pos¬ 
sible  ways  of  dealing  with  it.  The  latter 
actions  are  part  of  risk  analysis  and  risk 
handling,  but  often  begin  during  risk 
identification. 

Prioritization  is  the  ranking  of  risk  events 
to  determine  the  order  of  importance.  It 
serves  as  the  basis  for  risk-handling  actions. 
Prioritization  is  part  of  risk  analysis. 

Integrated  Product  Teams  (IPTs)  typically 
perform  risk  assessments  in  a  decentralized 
risk  management  organization  as  de¬ 
scribed  in  Paragraph  4.4.  If  necessary,  the 
team  may  be  augmented  by  people  from 
other  program  areas  or  outside  experts. 
Paragraph  5.4,  Risk  Assessment  Tech¬ 
niques,  elaborates  on  this  for  each  of  the 
described  assessment  techniques. 

2.6.3  Timing  of  Risk  Assessments 

The  assessment  process  begins  during  the 
last  half  of  Phase  0,  Concept  Exploration, 
and  continues  throughout  the  subsequent 
phases.  The  PMO  should  continually  re¬ 
assess  the  program  at  increasing  levels  of 
detail  as  the  program  progresses  through 
the  acquisition  phases  and  more  informa¬ 
tion  becomes  available.  There  are,  how¬ 
ever,  times  when  events  may  require  new 
assessments,  i.e.,  a  major  change  in  the  ac¬ 
quisition  strategy.  Paragraph  2.5.2  lists 
other  events  that  could  cause  risk  assess¬ 
ments  to  be  performed. 
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2.6.4  Conducting  Risk  Assessments 

There  is  no  standard  approach  to  assess¬ 
ing  risk  because  methods  vary  according 
to  the  technique  employed,  the  phase  of  the 
program,  and  the  nature  of  the  program 
itself;  however,  some  top-level  actions  are 
typically  common  to  all  methods.  They  are 
grouped  in  Figure  2-4  into  pre-risk  assess¬ 
ment  activities,  risk  identification  activities, 
and  risk  analysis  activities.  Each  risk  cat¬ 
egory  or  area,  e.g.,  cost,  schedule,  and  per¬ 
formance,  includes  a  core  set  of  assessment 


tasks  and  is  related  to  the  other  two  cat¬ 
egories.  This  relationship  requires  support¬ 
ive  analysis  among  areas  to  ensure  the  in¬ 
tegration  of  the  assessment  process.  For  ex¬ 
ample,  a  technical  assessment  probably 
should  include  a  cost  and  schedule  analy¬ 
sis  in  determining  the  technical  risk  impact. 
The  results  of  the  assessments,  normally 
conducted  by  IPTs  follow: 

Performance/Technical  Assessment  (In¬ 
cludes  technical  areas  of  risk  shown  in 
Paragraph  2.4.2.) 


Figure  2-4.  Risk  Assessment 
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•  Provides  technical  foundation, 

•  Identifies  and  describes  program 
risks,  i.e.,  threat,  technology,  design,  manu¬ 
facturing,  etc., 

•  Prioritizes  risks  with  relative  or  quan¬ 
tified  weight  for  program  impact, 

•  Analyzes  risks  and  relates  them  to 
other  internal  and  external  risks, 

•  Quantifies  associated  program  activi¬ 
ties  with  both  time  duration  and  resources, 

•  Quantifies  inputs  for  schedule  assess¬ 
ment  and  cost  estimate, 

•  Documents  technical  basis  and  risk 
definition  for  the  risk  assessment. 

Schedule  Assessment 

•  Evaluates  baseline  schedule  inputs, 

•  Incorporates  technical  assessment  and 
schedule  uncertainty  inputs  to  program 
schedule  model, 

•  Evaluates  impacts  to  program  sched¬ 
ule  based  on  technical  team  assessment, 

•  Performs  schedule  analysis  on  pro¬ 
gram  integrated  master  schedule, 

•  Quantifies  schedule  excursions  re¬ 
flecting  effects  of  cost  risks,  including  re¬ 
source  constraints, 

•  Provides  Government  schedule  as¬ 
sessment  for  cost  analysis  and  fiscal  year 
planning, 

•  Reflects  technical  foundation,  activity 
definition*  and  inputs  from  technical  and 
cost  areas, 

•  Documents  schedule  basis  and  risk 
impacts  for  the  risk  assessment. 

Cost  Estimate  and  Assessment 

•  Builds  on  technical  and  schedule  as¬ 
sessment  results. 


•  Translates  technical  and  schedule 
risks  into  cost, 

•  Derives  cost  estimate  by  integrating 
technical  risk  and  schedule  risk  impacts 
with  resources, 

•  Establishes  budgetary  requirements 
consistent  with  fiscal  year  planning, 

•  Determines  if  the  phasing  of  funds 
supports  technical  and  acquisition 
approach, 

•  Provides  program  cost  excursions 
from: 

—  Near-term  budget  execution 
impacts, 

—  External  budget  changes  and 
constraints. 

•  Documents  cost  basis  and  risk 
impacts. 

2.6.4.1  Pre-Risk  Assessment  Activities. 

The  Risk  Management  Plan  may  describe 
the  actions  that  compose  this  activity. 
Typically,  a  program-level  IPT  may  con¬ 
duct  a  quick-look  assessment  of  the  pro¬ 
gram  to  identify  the  need  for  technical  ex¬ 
perts  (who  are  not  part  of  the  team)  and 
to  examine  areas  that  appear  most  likely 
to  contain  risk.  The  program's  risk  coor¬ 
dinator,  or  an  outside  expert,  may  train 
the  IPTs,  focusing  on  the  program's  risk 
strategy,  definitions,  suggested  tech¬ 
niques,  documentation,  and  reporting  re¬ 
quirements.  Paragraph  4.9,  Risk  Manage¬ 
ment  Training,  provides  some  suggestions 
for  training. 

2.6.4.2  Risk  Identification  Activity.  To 

identify  risk  events,  IPTs  should  break 
down  program  elements  to  a  level  where 
they,  or  subject-matter  experts,  can  per¬ 
form  valid  assessments.  The  information 
necessary  to  do  this  varies  according  to 
the  phase  of  the  program.  During  the  early 
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phases,  requirement,  threat  documents, 
and  acquisition  plans  may  be  the  only 
program-specific  data  available.  They 
should  be  analyzed  to  identify  events  that 
may  have  adverse  consequences.  A  use¬ 
ful  initial  identification  exercise  is  to  per¬ 
form  a  mission  profile  for  the  system  as 
suggested  in  DoD  4245. 7-M,  Transition 
from  Development  to  Production.  Using  this 
methodology,  the  developer  creates  a 
functional  and  environmental  profile  for 
the  system  and  examines  the  low-level  re¬ 
quirements  that  the  system  must  meet  to 
satisfy  its  mission  requirements.  The  IPTs 
may  then  study  these  requirements  to  de¬ 
termine  which  are  critical.  For  example, 
in  an  aircraft  profile,  it  may  be  apparent 
that  high  speed  is  critical.  If  the  speed  re¬ 
quirement  is  close  to  that  achieved  by  ex¬ 
isting  aircraft,  this  may  not  be  a  concern. 
However,  if  the  speed  is  greater  than  that 
achieved  by  today's  aircraft,  it  may  be  a 
critical  risk  area.  Since  aircraft  speed  de¬ 
pends,  among  other  things,  on  weight  and 
engine  thrust,  it  would  be  desirable  to 
enlist  the  help  of  a  materials  expert  to  ad¬ 
dress  weight  and  an  engine  expert  to 
assess  engine-associated  risk. 

Another  method  of  decomposition  is  to 
create  a  WBS  as  early  as  possible  in  a 


program.  Figure  2-5  is  a  simple  example 
of  a  decomposition  based  on  the  WBS  for 
an  aircraft.  The  figure  shows  an  important 
requirement  of  the  decomposition  process, 
the  establishment  of  goals  (e.g.,  don't  ex¬ 
ceed  the  weight  budget  or  objective).  Risk 
events  are  determined  by  matching  each 
WBS  element  and  process  to  sources  or  ar¬ 
eas  of  risk.  Risk  areas/sources  are  de¬ 
scribed  in  Paragraph  2.4.2  and  Table  4-2. 

During  decomposition,  risk  events  are 
identified  from  experience,  brainstorming, 
lessons  learned  from  similar  programs,  and 
guidance  contained  in  the  risk  manage¬ 
ment  plan.  A  structured  approach  previ¬ 
ously  discussed  matches  each  WBS  ele¬ 
ment  and  process  in  terms  of  sources  or 
areas  of  risk.  The  examination  of  each  event 
is  an  exploratory  exercise  to  identify  the 
critical  risks.  The  investigation  may  show 
that  risks  are  interrelated.  For  example,  the 
weight  of  an  aircraft  affects  its  speed,  but 
also  impacts  the  payload,  range,  and  fuel 
requirements.  These  have  design  and  lo¬ 
gistics  consequences  and  may  even  affect 
the  number  of  aircraft  that  must  be  pro¬ 
cured  to  meet  objectives. 

Critical  risks  need  to  be  documented  as 
specified  in  the  Risk  Management  Plan  and 
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may  include  the  scenario  that  causes  the  risk, 
planned  management  controls  and  actions, 
etc.  It  may  also  contain  an  initial  assessment 
of  the  consequences  to  focus  the  risk  assess¬ 
ment  effort.  A  risk  watch  list  should  be  ini¬ 
tiated  as  part  of  risk  identification.  It  is  re¬ 
fined  during  handling,  and  monitored/ 
updated  during  the  monitoring  phase. 

2.6.4.3  Risk  Analysis  Activity.  Analysis 
begins  with  a  detailed  study  of  the  criti¬ 
cal  risks  that  have  been  identified.  The  ob¬ 
jective  is  to  gather  enough  information 
about  the  risks  to  judge  the  probability  of 
occurrence  and  the  impact  on  cost,  sched¬ 
ule,  and  performance  if  the  risk  occurs. 

Impact  assessments  are  normally  subjective 
and  based  on  detailed  information  that 
may  come  from: 

•  Comparisons  with  similar  systems, 

•  Relevant  lessons-learned  studies, 

•  Experience, 

•  Results  from  tests  and  prototype 
development, 

•  Data  from  engineering  or  other  models, 

•  Specialist  and  expert  judgments, 

•  Analysis  of  plans  and  related 
documents, 

•  Modeling  and  simulation. 


•  Sensitivity  analysis  of  alternatives. 

Depending  on  the  particular  technique  and 
the  risk  being  analyzed,  some  supporting 
analysis  may  be  necessary,  i.e.,  analysis  of 
contractor  processes,  such  as  design,  engi¬ 
neering,  fault  tree  analysis,  engineering 
models,  simulation,  etc.  Analyses  provide 
the  basis  for  subjective  assessments. 

A  critical  aspect  of  risk  analysis  is  data  col¬ 
lection.  Two  primary  sources  of  data  are 
interviews  of  subject-matter  experts  and 
analogy  comparisons  with  similar  sys¬ 
tems.  Paragraph  5.4  contains  a  technique 
for  collecting  both  types  of  data  for  use  in 
support  of  the  techniques  listed  in  Table 
2-1.  Periodically,  sets  of  risks  need  to  be 
prioritized  in  preparation  for  risk  han¬ 
dling,  and  aggregated  to  support  pro¬ 
gram  management  reviews.  Paragraph 
5.5,  Risk  Prioritization,  describes  methods 
for  accomplishing  this. 

2.6.4.3.1  Risk  Rating  and  Prioritization/ 
Ranking 

Ratings  are  an  indication  of  the  potential 
impact  of  risks  on  a  program;  they  are  a 
measure  of  the  likelihood  of  an  event 
occurring  and  the  consequences  of  the 
event.  They  are  often  expressed  as  high, 
moderate,  and  low.  Risk  rating  and 
prioritization/ranking  are  considered  in¬ 
tegral  parts  of  risk  analysis. 


Risk  Assessment  Technique 

Applicable  Acquisition  Phases 

Applicable  Risk  Areas  & 
Processes 

Plan  Evaluation/Risk  Identification 

All  phases 

Program  Plans  and  critical  com¬ 
munications  with  the  developer 

Product  (WBS)  Risk  Assessment 

All  phases  starting  with  the 
completion  of  the  Contract  WBS 

All  critical  risk  areas  except  threat, 
requirements,  cost,  and  schedule 

Process  (DoD  4265.7-M)  Risk 
Assessment 

All  phases  but  mainly  EMD 

All  critical  risk  processes 

Cost  Risk  Assessment 

All  phases 

Cost  critical  risk  areas 

Schedule  Risk  Assessment 

All  phases 

Schedule  critical  risk  areas 

Table  2-1.  Risk  Assessment  Approaches 
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A  group  of  experts,  who  are  familiar  with 
each  risk  area  (e.g.,  design,  logistics,  pro¬ 
duction,  etc.)  and  product  WBS  element 
risk  ratings,  are  best  qualified  to  determine 
risk  ratings.  They  should  identify  rating 
criteria  for  review  by  the  PMO,  who  in¬ 
cludes  them  in  the  Risk  Management  Plan. 
In  most  cases,  the  criteria  will  be  based  on 
the  experience  of  the  experts,  as  opposed 
to  mathematically  derived  and  should  es¬ 
tablish  levels  of  likelihood  and  conse¬ 
quences  that  will  provide  a  range  of  possi¬ 
bilities  large  enough  to  distinguish  differ¬ 
ences  in  risk  ratings.  At  the  program  level, 
consequences  should  be  expressed  in  terms 
of  impact  on  cost,  schedule  and  perfor¬ 
mance.  Tables  2-2  and  2-3  are  examples  of 
likelihood  and  consequence  criteria,  and 
Table  2-4  contains  an  example  of  overall 
risk  rating  criteria,  which  considers  both 
likelihood  and  consequences.  Table  2-5  pro¬ 
vides  a  sample  format  for  presenting  risk 
ratings. 


Level 

What  is  the  Likelihood  the  Risk 
Event  Will  Happen? 

a 

Remote 

b 

Unlikely 

c 

Likely 

d 

Highly  likely 

e 

Near  certainty 

Table  2-2.  Likelihood  Criteria  (Example) 


Rating 

Description 

High 

Moderate 

Low 

Major  disruption  likely 
Some  disruption 
Minimum  impact 

Table  2-4.  Overall  Risk  Rating  Criteria 
(Example) 


Using  these  risk  ratings,  PMs  can  identify 
events  requiring  priority  management 
(high  or  moderate  risk  likelihood  or  con¬ 
sequences).  The  document  prioritizing  the 
risk  events  is  called  a  Watch  List.  Risk  rat¬ 
ings  also  help  to  identify  the  areas  that 
should  be  reported  within  and  outside  the 
PMO,  e.g.,  milestone  decision  reviews. 
Thus,  it  is  important  that  the  ratings  be 
portrayed  as  accurately  as  possible. 

A  simple  method  of  representing  the  risk 
rating  for  risk  events  is  shown  in  Figure  2- 
6.  In  this  example,  the  PM  has  defined  lev¬ 
els  high,  moderate,  and  low  for  the  various 
combinations  of  likelihood  and  conse¬ 
quences. 

There  is  a  common  tendency  to  attempt 
to  develop  a  single  number  to  portray  the 
risk  associated  with  a  particular  event. 
This  approach  may  be  suitable  if  both  like¬ 
lihood  (probability)  and  consequences 
have  been  quantified  using  compatible 
cardinal  scales  or  calibrated  ordinal  scales 


Level 

Given  the  Risk  Is  Realized,  What  Is  the  Magnitude  of  the  Impact? 

Performance 

Schedule 

Cost 

1 

Minimal  or  no  impact 

Minimal  or  no  impact 

Minimal  or  no  impact 

2 

Acceptable  with  some 
reduction  in  margin 

Additional  resources 
required;  able  to  meet 
need  dates 

<5% 

3 

Acceptable  with  significant 
reduction  in  margin 

Minor  slip  in  key  milestones; 
not  able  to  meet  need  date 

5-7% 

4 

Acceptable;  no  remaining 
margin 

Major  slip  in  key  milestone 
or  critical  path  impacted 

7-10% 

5 

Unacceptable 

Can’t  achieve  key  team  or 
major  program  milestone 

>10% 

Table  2-3.  Consequences  Criteria  (Example) 
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Priority 

Area 1 
Process 

Location 

Title 

Likeli¬ 

hood 

Conse¬ 

quence 

Time 

Constraints 

1 

Design 

WBS  3.1 

Design 

completeness 

High 

High 

1-2  months 

2 

3 

Table  2-5.  Risk  Ratings  (Example) 
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Figure  2-6.  Overall  Risk  Rating 
(Example) 


whose  scale  levels  have  been  determined 
using  accepted  procedures  (e.g..  Analyti¬ 
cal  Hierarchy  Process).  In  such  a  case, 
mathematical  manipulation  of  the  values 
may  be  meaningful  and  provide  some 
quantitative  basis  for  the  ranking  of  risks. 

In  most  cases,  however,  risk  scales  are  ac¬ 
tually  just  raw  (uncalibrated)  ordinal 
scales,  reflecting  only  relative  standing  be¬ 
tween  scale  levels  and  not  actual  numerical 
differences.  Any  mathematical  operations 
performed  on  results  from  uncalibrated  or¬ 
dinal  scales,  or  a  combination  of  uncali¬ 
brated  ordinal  and  cardinal  scales,  can  pro¬ 
vide  information  that  will  at  best  be  mis¬ 
leading,  if  not  completely  meaningless,  re¬ 
sulting  in  erroneous  risk  ratings.  Hence, 
mathematical  operations  should  generally  not 
be  performed  on  scores  derived  from  uncali¬ 
brated  ordinal  scales.  (Note:  risk  scales  that 
are  expressed  as  decimal  values  (e.g.,  a  5 
level  scale  with  values  0.2, 0.4, 0.6, 0.8  and 
1.0)  still  retain  the  ordinal  scale  limitations 
discussed  above.) 


One  way  to  avoid  this  situation  is  to  sim¬ 
ply  show  each  risk  event's  likelihood  and 


consequences  separately,  with  no  attempt 
to  combine  multiple  risks.  Other  factors 
that  may  significantly  contribute  to  the  risk 
rating  or  prioritization  of  risk  events,  such 
as  time  sensitivity  or  resource  availability, 
can  also  be  shown.  The  prioritization  or 
ranking  should  also  be  done  based  on  a 
structural  risk  rating  approach  (e.g.  Figure 
2-6)  coupled  with  expert  opinion  and  ex¬ 
perience.  Prioritization  or  ranking  is 
achieved  through  integration  of  risk  events 
from  lower  to  higher  levels.  This  means 
that  the  effect  of  risk  at  lower  WBS  ele¬ 
ments  needs  to  be  reflected  cumulatively 
at  the  top  or  system  level. 

2.7  RISK  HANDLING 

2.7.1  Purpose  of  Risk  Handling 

Risk  handling  includes  specific  methods 
and  techniques  to  deal  with  known  risks 
and  a  schedule  for  accomplishing  tasks, 
identifies  who  is  responsible  for  the  risk 
area,  and  provides  an  estimate  of  the  cost 
and  schedule  associated  with  handling  the 
risk,  if  any.  It  involves  planning  and  ex¬ 
ecution  with  the  objective  of  handling  risks 
at  an  acceptable  levels.  The  IPTs  that  as¬ 
sess  risk  should  begin  the  process  to  iden¬ 
tify  and  evaluate  handling  approaches  to 
propose  to  the  PM,  who  selects  the  appro¬ 
priate  ones  for  implementation. 

2.7.2  Risk-Handling  Process 

The  risk-handling  phase  must  be  compat¬ 
ible  with  the  risk  management  plan  and 
any  additional  guidance  the  PM  provides. 
Paragraph  5.3  describes  a  technique  that 
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concentrates  on  planning.  A  critical  part 
planning  involves  refining  and  selecting  of 
the  most  appropriate  handling  options. 

The  IPTs  that  evaluate  the  handling  options 
may  use  the  following  criteria  as  a  starting 
point  for  assessment: 

•  Can  the  option  be  feasibly  imple¬ 
mented  and  still  meet  the  user's  needs? 

•  What  is  the  expected  effectiveness  of 
the  handling  option  in  reducing  program 
risk  to  an  acceptable  level? 

•  Is  the  option  affordable  in  terms  of 
dollars  and  other  resources  (e.g.,  use  of 
critical  materials,  test  facilities,  etc.)? 

•  Is  time  available  to  develop  and 
implement  the  option,  and  what  effect  does 
that  have  on  the  overall  program  schedule? 

•  What  effect  does  the  option  have  on 
the  system's  technical  performance? 

Risk-handling  options  can  include  risk 
avoidance,  risk  control,  risk  transfer,  risk 
assumption.  Although  the  control  risk¬ 
handling  option  is  commonly  used  in  de¬ 
fense  programs,  it  should  not  automatically 
be  chosen.  All  four  options  should  be 
evaluated  and  the  best  one  chosen  for  a 
given  risk  issue. 

Risk  control  does  not  attempt  to  eliminate 
the  source  of  the  risk  but  seeks  to  reduce 
or  mitigate  the  risks.  It  monitors  and  man¬ 
ages  the  risk  in  a  manner  that  reduces  the 
likelihood  and/or  consequence  of  its  oc¬ 
currence  or  minimizes  the  risk's  effect  on 
the  program.  This  option  may  add  to  the 
cost  of  a  program;  however,  the  selected 
approach  should  provide  an  optional  risk 
among  the  candidate  approaches  of  risk  re¬ 
duction,  cost  effectiveness,  and  schedule 
impact.  A  sampling  is  listed  below  of  the 
types  of  risk  control  actions  available  to  the 


PMO.  Paragraph  5.6.2  discusses  them  in 
more  detail. 

•  Multiple  Development  Efforts.  Create 
competing  systems  in  parallel  that  meet  the 
same  performance  requirements. 

•  Alternative  Design.  Create  a  backup 
design  option  that  uses  a  lower  risk 
approach. 

•  Trade  Studies.  Arrive  at  a  balance  of 
engineering  requirements  in  the  design  of 
a  system. 

•  Early  Prototyping.  Build  and  test  pro¬ 
totypes  early  in  the  system  development. 

•  Incremental  Development.  Design 
with  the  intent  of  upgrading  system  parts  in 
the  future. 

•  Technology  Maturation  Efforts.  Nor¬ 
mally,  technology  maturation  is  used  when 
the  desired  technology  will  replace  an  ex¬ 
isting  technology  which  is  available  for  use 
in  the  system. 

•  Robust  Design.  This  approach,  while 
it  could  be  more  costly,  uses  advanced  de¬ 
sign  and  manufacturing  techniques  that 
promote  quality  through  design. 

•  Reviews,  Walk  throughs,  and  In¬ 
spections.  These  three  actions  can  be  used 
to  reduce  the  likelihood  and  potential  con¬ 
sequences  of  risks  through  timely  assess¬ 
ment  of  actual  or  planned  events. 

•  Design  of  Experiments.  This  engi¬ 
neering  tool  identifies  critical  design  fac¬ 
tors  that  are  sensitive,  therefore  potentially 
high  risk,  to  achieve  a  particular  user 
requirement. 

•  Open  Systems.  Carefully  selected 
commercial  specifications  and  standards 
whose  use  can  result  in  lower  risks. 
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•  Use  of  Standard  Items/Software  Re¬ 
use.  Use  of  existing  and  proven  hardware 
and  software,  where  applicable,  can  sub¬ 
stantially  reduce  risks. 

•  Two-Phase  Engineering  and  Manu¬ 
facturing  Development.  Incorporation  of 
a  formal  risk  reduction  phase  at  the  initial 
part  of  EMD.  This  technique  is  sometimes 
used  instead  of  a  formal  PDRR  phase  if  risk 
is  moderate  or  low. 

•  Use  of  Mock-ups.  The  use  of  mock- 
ups,  especially  man-machine  interface 
mock-ups,  can  be  used  to  conduct  early  ex¬ 
ploration  of  design  options. 

•  Modeling/Simulation.  Modeling  and 
simulation  can  be  used  to  investigate  vari¬ 
ous  design  options  and  system  requirement 
levels. 

•  Key  Parameter  Control  Boards.  The 

practice  of  establishing  a  control  board  for 
a  parameter  may  be  appropriate  when  a 
particular  feature  (such  as  system  weight) 
is  crucial  to  achieving  the  overall  program 
requirements. 

•  Manufacturing  Screening.  For  pro¬ 
grams  in  Engineering  and  Manufacturing 
Development  (EMD),  various  manufactur¬ 
ing  screens  (including  environmental  stress 
screening  (ESS))  can  be  incorporated  into 
test  article  production  and  low  rate  initial 
production  (LRIP)  to  identify  deficient 
manufacturing  processes.  ESS  is  a  manu¬ 
facturing  process  for  stimulating  parts  and 
workmanship  defects  in  electronic  assem¬ 
blies  and  units. 

As  you  can  see,  there  are  numerous  means 
that  can  be  used  to  actively  control  risks. 

Risk  avoidance  involves  a  change  in  the 
concept,  requirements,  specifications, 
and/or  practices  that  reduce  risk  to  an 


acceptable  level.  Simply  stated,  it  elimi¬ 
nates  the  sources  of  high  or  possibly  me¬ 
dium  risk  and  replaces  them  with  a  lower 
risk  solution  and  may  be  supported  by  a 
cost/benefit  analysis.  Generally,  this 
method  may  be  done  in  parallel  with  the 
up-front  requirements  analysis,  supported 
by  cost /requirement  trade  studies,  which 
can  include  cost-as-an-independent-vari- 
able  (CAIV)  trades. 

Risk  Assumption.  Risk  assumption  is  an 
acknowledgment  of  the  existence  of  a  par¬ 
ticular  risk  situation  and  a  conscious  deci¬ 
sion  to  accept  the  associated  level  of  risk, 
without  engaging  in  any  special  efforts  to 
control  it.  However,  a  general  cost  and 
schedule  reserve  may  be  set  aside  to  deal 
with  any  problems  that  may  occur  as  a  re¬ 
sult  of  various  risk  assumption  decisions. 
This  method  recognizes  that  not  all  identi¬ 
fied  program  risks  warrant  special  han¬ 
dling;  as  such,  it  is  most  suited  for  those 
situations  that  have  been  classified  as  low 
risk.  The  key  to  successful  risk  assumption 
is  twofold: 

•  Identify  the  resources  (time,  money, 
people,  etc.)  needed  to  overcome  a  risk  if 
it  materializes.  This  includes  identifying 
the  specific  management  actions  (such  as 
retesting,  additional  time  for  further  design 
activities)  that  may  occur. 

•  Ensure  that  necessary  administrative 
actions  are  taken  to  identify  a  management 
reserve  to  accomplish  those  management 
actions. 

Risk-handling  options  have  broad  cost  im¬ 
plications.  The  magnitude  of  these  costs  are 
circumstance-dependent.  The  approval 
and  funding  of  handling  options  should 
be  part  of  the  process  that  establishes  the 
program  cost  and  performance  goals.  This 
should  normally  be  done  by  the  Program- 
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Level  Risk  Management  IPT  or  Risk  Man¬ 
agement  Board.  The  selected  handling  op¬ 
tion  should  be  included  in  the  program's 
acquisition  strategy. 

Once  the  acquisition  strategy  includes  risk¬ 
handling  approaches,  the  PMO  can  derive 
the  schedule  and  identify  cost,  schedule, 
and  performance,  impacts  to  the  basic 
program. 

Risk  Transfer.  This  action  may  reallocate 
risk  during  the  concept  development  and 
design  processes  from  one  part  of  the  sys¬ 
tem  to  another,  thereby  reducing  the  over¬ 
all  system  risk,  or  re-distributing  risks  be¬ 
tween  the  Government  and  the  prime  con¬ 
tractor  or  within  Government  agencies;  or 
between  members  of  the  contractor  team. 
It  is  an  integral  part  of  the  functional  analy¬ 
sis  process.  Risk  transfer  is  a  form  of  risk 
sharing  and  not  risk  abrogation  on  the  part 
of  the  Government,  and  it  may  influence 
cost  objectives.  An  example  is  the  transfer 
of  a  function  from  hardware  implementa¬ 
tion  to  software  implementation  or  vice 
versa.  The  effectiveness  of  risk  transfer  de¬ 
pends  on  the  use  of  successful  system  de¬ 
sign  techniques.  Modularity  and  functional 
partitioning  are  two  design  techniques  that 
support  risk  transfer.  In  some  cases,  risk 
transfer  may  concentrate  risk  areas  in  one 
area  of  the  design.  This  allows  manage¬ 
ment  to  focus  attention  and  resources  on 
that  area. 

2.8  RISK  MONITORING 

The  monitoring  process  systematically 
tracks  and  evaluates  the  effectiveness  of 
risk-handling  actions  against  established 
metrics.  Monitoring  results  may  also  pro¬ 
vide  a  basis  for  developing  additional  han¬ 
dling  options  and  identifying  new  risks. 
The  key  to  the  monitoring  process  is  to  es¬ 
tablish  a  cost,  schedule,  and  performance 


management  indicator  system  over  the  en¬ 
tire  program  that  the  PM  uses  to  evaluate 
the  status  of  the  program.  The  indicator 
system  should  be  designed  to  provide  early 
warning  of  potential  problems  to  allow 
management  actions.  Risk  monitoring  is 
not  a  problem-solving  technique,  but 
rather,  a  proactive  technique  to  observe  the 
results  of  risk  handling  and  identify  new 
risks.  Some  monitoring  techniques  can  be 
adapted  to  become  part  of  a  risk  indicator 
system: 

•  Test  and  Evaluation  (T&E).  A  well- 
defined  (T&E)  program  is  a  key  element 
in  monitoring  the  performance  of  selected 
risk-handling  options  and  developing  new 
risk  assessments. 

•  Test-Analyze-and-Fix  (TAAF).  TAAF 
is  the  use  of  a  period  of  dedicated  testing 
to  identify  and  correct  deficiencies  in  a 
design. 

•  Demonstration  Events.  Demonstra¬ 
tion  events  are  points  in  the  program  (nor¬ 
mally  tests)  that  determine  if  risks  are  be¬ 
ing  successfully  abated. 

•  Earned  Value  (EV).  This  uses  stan¬ 
dard  DoD  cost/ schedule  data  to  evaluate 
a  program's  cost  and  schedule  performance 
in  an  integrated  fashion.  As  such,  it  pro¬ 
vides  a  basis  to  determine  if  risk-handling 
actions  are  achieving  their  forecasted 
results. 

•  Technical  Performance  Measure¬ 
ment  (TPM).  TPM  is  a  product  design 
assessment  which  estimates,  through  en¬ 
gineering  analysis  and  tests,  the  values 
of  essential  performance  parameters  of 
the  current  design  as  effected  by  risk¬ 
handling  actions. 

•  Program  Metrics.  These  are  used  for 
formal,  periodic  performance  assessments 
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of  the  various  development  processes, 
evaluating  how  well  the  system  develop¬ 
ment  process  is  achieving  its  objective.  This 
technique  can  be  used  to  monitor  correc¬ 
tive  actions  that  emerged  from  an  assess¬ 
ment  of  the  critical  risk  processes. 

•  Process  Proofing.  Similar  to  Program 
Metrics,  but  aimed  at  manufacturing  and 
support  processes  which  are  critical  to 
achieving  system  requirements.  Proofing 
simulates  actual  production  environments 
and  conditions  to  insure  repeatedly  con¬ 
forming  hardware  and  software. 

•  Schedule  Performance  Monitoring. 

This  is  the  use  of  program  schedule  data 
to  evaluate  how  well  the  program  is  pro¬ 
gressing  to  completion. 

Paragraph  5.7  describes  several  monitor¬ 
ing  techniques,  e.g.,  earned  value. 

The  indicator  system  and  periodic  reas¬ 
sessments  of  program  risk  should  provide 
the  PMO  with  the  means  to  incorporate 
risk  management  into  the  overall  program 
management  structure. 

2.9  RISK  DOCUMENTATION 

A  primary  criteria  for  successful  manage¬ 
ment  is  formally  documenting  the  ongoing 
risk  management  process.  This  is  impor¬ 
tant  because: 

•  It  provides  the  basis  for  program  as¬ 
sessments  and  updates  as  the  program 
progresses. 

•  Formal  documentation  tends  to  en¬ 
sure  more  comprehensive  risk  assessments 
than  if  it  is  not  documented. 

•  It  provides  a  basis  for  monitoring 
risk-handling  actions  and  verifying  the 
results. 


•  It  provides  program  background  ma¬ 
terial  for  new  personnel. 

•  It  is  a  management  tool  for  the  execu¬ 
tion  of  the  program. 

•  It  provides  the  rationale  for  program 
decisions. 

The  documentation  should  be  done  by 
those  responsible  for  planning  and  collect¬ 
ing  and  analyzing  data,  i.e.,  IPT  level  in 
most  cases. 

Risk  management  reports  vary  depending 
on  the  size,  nature,  and  phase  of  the  pro¬ 
gram.  Examples  of  some  risk  management 
documents  and  reports  that  may  be  useful 
to  a  PM  are: 

•  Risk  Management  Plan, 

•  Risk  information  form, 

•  Risk  assessment  report, 

•  Risk  handling  priority  list, 

•  Risk  handling  plan  of  action, 

•  Aggregated  risk  list, 

•  Risk  monitoring  documentation: 

—  Program  metrics, 

—  Technical  reports, 

—  Earned  value  reports, 

—  Watch  list, 

—  Schedule  performance  report, 

—  Critical  risk  processes  reports. 

Most  PMOs  can  devise  a  list  of  standard 
reports  that  will  satisfy  their  needs  most 
of  the  time;  however,  since  there  will  al¬ 
ways  be  a  need  for  ad  hoc  reports  and  brief¬ 
ing  and  assessments,  it  is  advisable  to  store 
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risk  information  in  a  management  infor¬ 
mation  system  (MIS).  This  allows  you  to 
derive  standard  reports  and  create  of  ad 
hoc  reports,  as  needed.  Paragraphs  4.8  and 
5.8  discuss  an  MIS  to  support  a  risk  man¬ 
agement  program. 

Acquisition  reform  discourages  Govern¬ 
ment  oversight;  therefore,  formal 
contractor-produced  risk  documentation 
may  not  be  available  for  most  programs. 
However,  program  insight  is  encouraged, 
and  PMOs  can  obtain  information  about 
program  risk  from  contractor  internal 
documentation  such  as: 

•  Risk  Management  Policy  and  Pro¬ 
cedures.  This  is  a  description  of  the 
contractor's  corporate  policy  for  the  man¬ 
agement  of  risk.  The  procedures  describe 
the  methods  for  risk  identification,  analy¬ 
sis,  handling,  monitoring,  and  documen¬ 
tation.  It  should  provide  the  baseline  plan¬ 
ning  document  for  the  contractor's  ap¬ 
proach  to  risk  management. 


•  Corporate  Policy  and  Procedures 
Documents.  Corporations  have  policy 
and  procedures  documents  that  address 
the  functional  areas  that  are  critical  to  the 
design,  engineering,  manufacture,  test 
and  evaluation,  quality,  configuration 
control,  manufacture,  etc.,  of  a  system. 
These  documents  are  based  on  what  the 
company  perceives  as  best  practices,  and 
although  they  may  not  specifically  ad¬ 
dress  risk,  deviation  from  these  policies 
represents  risk  to  a  program.  Internal  com¬ 
pany  reports  that  address  how  well  pro¬ 
grams  comply  with  policy  may  be  re¬ 
quired  and  will  provide  valuable  infor¬ 
mation. 

•  Risk  Monitoring  Report.  Contrac¬ 
tors  should  have  internal  tracking  metrics 
and  reports  for  each  moderate-  or  high- 
risk  item.  These  metrics  may  be  used  to 
determine  the  status  of  risk  reduction 
programs. 


22 


3 

RISK  MANAGEMENT  AND 
DOD  ACQUISITION  PROCESS 


3.1  INTRODUCTION 

This  Chapter  discusses  the  relationship  be¬ 
tween  risk  and  the  acquisition  process,  de¬ 
scribes  how  risk  is  considered  in  design  of 
the  Acquisition  Plan,  and  expresses  the 
need  to  consider  risk  as  early  in  the  pro¬ 
gram  as  possible.  Appendix  A  is  a  sum¬ 
mary  of  the  risk  management  requirements 
that  are  contained  in  DoDD  5000.1  and 
DoD  5000.2-R,  DoD  5000.4,  and  DoD 
5000.4-M. 

3.2  OVERVIEW 

The  DoD  acquisition  process  for  the  man¬ 
agement  of  programs  consists  of  a  series 
of  phases  designed  to  reduce  risk,  ensure 
affordability,  and  provide  adequate  infor¬ 
mation  for  decision  making.  Acquisition  of¬ 
ficials  are  encouraged  to  tailor  programs 
to  eliminate  phases  or  activities  that  result 
in  little  payoff  in  fielding  time  or  cost  sav¬ 
ings.  To  effectively  tailor  a  program,  one 
needs  to  understand  the  risks  present  in 
the  program  and  to  develop  a  plan  for  man¬ 
aging  these  risks.  DoD  policy  calls  for  the 
continual  assessment  of  program  risks,  be¬ 
ginning  with  the  initial  phase  of  an  acqui¬ 
sition  program,  and  the  development  of 
management  approaches  before  any  deci¬ 
sion  is  made  to  enter  all  subsequent  phases. 

The  application  of  risk  management  pro¬ 
cesses  (planning,  assessment,  identifica¬ 
tion,  analysis,  handling,  and  monitoring) 


is  particularly  important  during  Phase  0 
of  any  program,  when  alternatives  are 
evaluated,  program  objectives  are  estab¬ 
lished,  and  the  acquisition  strategy  is  de¬ 
veloped.  All  of  these  activities  require  ac¬ 
ceptance  of  some  level  of  risk  and  devel¬ 
opment  of  plans  to  manage  the  risk. 

As  a  program  evolves  into  subsequent 
phases,  the  nature  of  the  risk  management 
effort  will  change.  New  assessments  will 
be  built  on  previous  ones.  Risk  areas  will 
become  more  specific  as  the  system  is 
defined. 

Risk  management  should  also  be  an  inte¬ 
gral  part  of  any  Source  Selection  process, 
from  RFP  preparation,  through  proposal 
evaluation,  and  after  contract  award. 
Throughout  the  program  life,  IPTs  will  play 
a  key  role  in  risk  management  activities. 

3.3  DOD  ACQUISITION  PROCESS 

The  phases  and  milestones  of  the  acquisi¬ 
tion  process  provide  a  streamlined  struc¬ 
ture  that  emphasizes  risk  management  and 
affordability.  The  phases  are  a  logical 
means  of  progressively  translating 
broadly-stated  mission  needs  into  well- 
defined  system-specific  requirements,  and 
ultimately  into  operationally  effective,  suit¬ 
able,  and  survivable  systems.  It  is  impor¬ 
tant  to  remember  that  the  term  "system" 
includes  hardware,  software,  and  the  hu¬ 
man  element.  Each  phase  is  designed. 
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among  other  things,  to  manage  risks.  Mile¬ 
stones  are  points  in  time  that  allow  deci¬ 
sion  makers  to  evaluate  the  program 
status  and  determine  if  the  program 
should  proceed  to  the  next  phase.  The 
Milestone  Decision  Authority  (MDA)  and 
PM  tailor  milestones  and  phases  so  that 
each  milestone  decision  point  allows  as¬ 
sessment  of  program  status  and  the  op¬ 
portunity  to  review  plans  for  the  next 
phase  and  beyond.  The  MDA  should  ex¬ 
plicitly  address  program  risks  and  the 
adequacy  of  risk  management  planning 
during  the  milestone  reviews  and  estab¬ 
lish  exit  criteria  for  progression  to  the  next 
phase. 

The  contract  schedule  normally  allows 
time  for  milestone  decisions  before  spend¬ 
ing  begins  in  subsequent  phases  and 
should  also  permit  demonstration  of  the 
exit  criteria  in  time  to  support  the  milestone 
review.  There  are  exceptions  to  this — 
driven  by  funding  availability  and  option 
award  dates.  However,  the  objective  is  to 
provide  proper  fiscal  control  without  de¬ 
laying  the  acquisition  decisions  or  contracts 
while  adequately  considering  risk. 

The  acquisition  strategy  defines  the  busi¬ 
ness  and  technical  management  approach 
to  meet  objectives  within  program  con¬ 
straints  with  a  primary  goal  to  minimize 
the  time  and  cost  of  satisfying  a  valid 
need,  consistent  with  common  sense  and 
sound  business  practices.  A  PM  prepares 
a  preliminary  acquisition  strategy  at  Mile¬ 
stone  0  (that  includes  Phase  0  activities 
that  focus  on  identifying  risk  and  han¬ 
dling  options).  Later,  the  PM  updates  the 
strategy  to  support  each  milestone  deci¬ 
sion  by  describing  activities  and  events 
planned  for  the  upcoming  phase  and  re¬ 
lating  the  accomplishments  of  that  phase 
to  the  program's  overall,  long-term  objec¬ 
tives.  The  risk  associated  with  a  program 


will  significantly  influence  the  acquisition 
strategy. 

3.4  CHARACTERISTICS  OF  THE 
ACQUISITION  PROCESS 

The  acquisition  process  that  has  evolved 
can  be  characterized  in  terms  of  the  follow¬ 
ing  concepts  that  are  particularly  relevant 
to  the  management  of  risk  in  programs. 

3.4.1  Integrated  Product  and  Process 
Development  (IPPD) 

IPPD  integrates  all  acquisition  activities 
in  order  to  optimize  system  development, 
production,  and  deployment.  Key  to  the 
success  of  the  IPPD  concept  are  the  IPTs, 
which  are  composed  of  qualified  and  em¬ 
powered  representatives  from  all  appro¬ 
priate  functional  disciplines  who  work  to¬ 
gether  to  identify  and  resolve  issues.  As 
such,  IPTs  are  the  foundation  for  organiz¬ 
ing  for  risk  management. 

3.4.2  Continuous  Risk  Management 

PMs  should  focus  on  risk  management 
throughout  the  life  of  the  program,  not  just 
in  preparation  for  program  and  milestone 
reviews.  Program  risks  should  be  continu¬ 
ously  assessed,  and  the  risk-handling  ap¬ 
proaches  developed,  executed,  and  moni¬ 
tored  throughout  the  acquisition  process. 
Both  the  Government  and  contractors 
must  understand  risks  as  a  program 
progresses  through  the  various  phases  and 
milestone  decision  points,  and  must 
modify  the  management  strategy  and  plan 
accordingly.  While  specific  government 
and  contractors  risk  management  pro¬ 
cesses  may  likely  be  different,  it  is  impor¬ 
tant  that  each  party  have  a  common  and 
complete  set  of  process  steps  (regardless 
of  their  names),  and  be  able  to  exchange 
and  clearly  understand  the  other  party's 
risk  management  documentation. 
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3.4.3  Program  Stability 

Once  a  program  is  initiated,  program  sta¬ 
bility  is  a  top  priority.  Keys  to  creating  pro¬ 
gram  stability  are  realistic  investment 
planning  and  affordability  assessments. 
They  must  reflect  an  accurate  and  com¬ 
prehensive  understanding  of  existing  or 
expected  program  risks.  A  risk  manage¬ 
ment  strategy  must  be  developed  early  in 
the  process,  before  actually  initiating  the 
program  to  ensure  it  is  a  stable  one,  rec¬ 
ognizing  that  key  issues  affecting  program 
stability  may  be  external. 

3.4.4  Reduction  of  Life-Cycle  Costs 

DoD  considers  the  reduction  of  total  cost 
to  acquire  and  operate  systems  while 
maintaining  a  high  level  of  performance 
for  the  user  to  be  of  highest  priority.  This 
is  reflected,  in  part,  through  the  introduc¬ 
tion  of  the  "Cost  As  an  Independent  Vari¬ 
able"  (CAIV)  concept.  CAIV  entails  set¬ 
ting  aggressive,  realistic  cost  objectives 
early  in  an  acquisition  program  and  then 
managing  all  aspects  of  the  program  to 
achieve  those  objectives,  while  still  meet¬ 
ing  the  user's  performance  and  schedule 
needs.  Inherent  in  the  CAIV  concept  is 
the  realization  that  risks  must  be  under¬ 
stood,  taken,  and  managed  in  order  to 
achieve  cost,  schedule,  and  performance 
objectives.  An  understanding  of  risk  is  es¬ 
sential  to  setting  realistic  cost  objectives. 
The  PM  and  user  representatives  should 
identify  risk  and  cost  driving  require¬ 
ments  during  the  generation  of  the  Op¬ 
erational  Requirement  Document  (ORD) 
in  order  to  know  where  tradeoffs  may  be 
necessary. 

3.4.5  Event-Oriented  Management 

Event-oriented  management  requires  that 
decision  makers  base  their  decisions  on 


significant  events  in  the  acquisition  life 
cycle,  rather  than  on  arbitrary  calendar 
dates.  This  management  process  empha¬ 
sizes  effective  acquisition  planning  and 
embodies  sound  risk  management.  Deci¬ 
sions  to  proceed  with  a  program  should 
be  based  on  demonstration  of  perfor¬ 
mance,  through  test  and  evaluation,  and 
on  verification  that  program  risks  are 
well-understood  and  are  being  managed 
effectively.  Attainment  of  agreed-upon 
exit  criteria  is  an  indication  that  the  PMO 
is  managing  risk  effectively. 

3.4.6  Modeling  and  Simulation 

Properly  used,  models  and  simulations 
can  reduce  time,  resources,  and  acquisi¬ 
tion  risk  and  may  increase  the  quality  of 
the  systems  being  developed.  Users  of 
these  models  and  simulations  must  have 
a  good  understanding  of  their  capabilities 
and  limitations  and  their  applicability  to 
the  issues  being  addressed. 

From  a  risk  perspective,  modeling  and 
simulation  may  be  used  to  develop  alter¬ 
native  concepts  during  system  design; 
predict  performance  in  support  of  trade¬ 
off  studies;  evaluate  system  design  and 
support  preliminary  design  reviews  dur¬ 
ing  design  development;  predict  system 
performance  and  supplement  live  tests 
during  testing;  examine  the  military  value 
of  the  system;  determine  the  impact  of  de¬ 
sign  changes;  hone  requirements;  and  de¬ 
velop  life  cycle  support  requirements  and 
assessments. 

However,  a  key  limitation  through  mod¬ 
els  and  simulations  is  that  the  results  are 
only  as  accurate  and  certain  as  the  qual¬ 
ity  of  the  underlying  relationships  and 
input  data.  Blindly  believing  and  using 
the  output  from  models  and  simulations 
should  never  be  done. 
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3.5  RISK  MANAGEMENT 
ACTIVITIES  DURING 
ACQUISITION  PHASES 

Risk  management  activities  should  be  ap¬ 
plied  continuously  throughout  all  acquisi¬ 
tion  process  phases.  However,  because  of 
the  difference  in  available  information,  the 
level  of  application  and  detail  will  vary  for 
each  phase.  In  Phase  0,  management  fo¬ 
cuses  on  assessing  the  risks  in  the  alterna¬ 
tive  concepts  available  to  satisfy  users 
needs  and  on  planning  a  strategy  to  ad¬ 
dress  those  risks.  For  each  of  the  subse¬ 
quent  phases,  all  four  risk  management 
activities  may  be  applied  with  increasing 
focus  on  risk  handling  and  monitoring. 

The  PM  identifies  objectives,  alternatives, 
and  constraints  at  the  beginning  of  each 
phase  of  a  program  and  then  evaluates  al¬ 
ternatives,  identifies  sources  of  project  risk, 
and  selects  a  strategy  for  resolving  the  risks. 
The  PMO  updates  the  acquisition  strategy, 
risk  assessments,  and  other  aspects  of  pro¬ 
gram  planning,  based  on  analyses,  for  the 
phase  of  the  acquisition. 

Developers  should  become  involved  in  the 
risk  management  process  at  the  beginning, 
when  users  define  performance  require¬ 
ments,  and  continue  during  the  acquisition 
process  until  the  system  is  delivered.  The 
early  identification  and  assessment  of  criti¬ 
cal  risks  allow  PMs  to  formulate  handling 
approaches  and  to  streamline  the  program 
definition  and  the  RFP  around  critical 
product  and  process  risks. 

The  following  paragraphs  address  risk  man¬ 
agement  in  the  different  phases  in  more 
detail. 

3.5.1  Phase  0 

DoD  5000.2-R  describes  Phase  0  as  nor¬ 
mally  consisting  of  studies  that  define  and 


evaluate  the  feasibility  of  alternative  con¬ 
cepts  and  provide  the  basis  for  the  assess¬ 
ment  of  these  alternatives  in  terms  of  their 
advantages,  disadvantages,  and  risk  lev¬ 
els  at  the  Milestone  (MS)  I  decision  point. 
In  addition  to  providing  input  to  the  Analy¬ 
sis  of  Alternatives,  the  PM  develops  a  pro¬ 
posed  acquisition  program  baseline  (APB) 
and  exit  criteria  for  Phase  I. 

The  APB  documents  the  most  important 
performance,  cost,  and  schedule  objectives 
and  thresholds  for  the  selected  concepts. 
The  parameters  selected  are  such  that  a  re- 
evaluation  of  alternative  concepts  is  appro¬ 
priate  if  thresholds  are  not  met.  Exit  crite¬ 
ria  are  events  or  accomplishments  that  al¬ 
low  managers  to  track  progress  in  critical 
technical,  cost,  or  schedule  risk  areas.  They 
must  be  demonstrated  to  show  that  a  pro¬ 
gram  is  on  track. 

In  defining  alternative  concepts,  PMs  should 
pay  particular  attention  to  the  threat  and  the 
user's  requirements,  which  are  normally 
stated  in  broad  terms  at  this  time.  Risks  can 
be  introduced  if  the  requirements  are  not 
stable,  or  if  they  are  overly  restrictive  and 
contain  specific  technical  solutions.  Require¬ 
ments  can  also  be  significant  cost  and  sched¬ 
ule  risk  drivers  if  they  require  a  level  of  per¬ 
formance  that  is  difficult  to  achieve  within 
the  program  budget  and  time  constraints. 
Such  drivers  need  to  be  identified  as  early 
in  the  program  as  possible. 

The  acquisition  strategy  should  address  the 
known  risks  for  each  alternative  concept, 
and  the  plans  to  handle  them,  including 
specific  events  intended  to  control  the  risks. 
Similarly,  the  T&E  strategy  should  reflect 
how  T&E,  with  the  use  of  M&S,  will  be 
used  to  assess  risk  levels  and  identify  new 
or  suspected  risk  areas. 

A  risk  management  strategy,  derived  in  con¬ 
cert  with  the  acquisition  strategy,  should  be 
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developed  during  this  phase  and  revised 
and  updated  continually  throughout  the 
program.  This  strategy  should  include  risk 
management  planning  that  clearly  defines 
roles,  responsibilities,  authority,  and  docu¬ 
mentation  for  program  reviews,  risk  assess¬ 
ments,  and  risk  monitoring. 

3.5.2  Subsequent  Phases 

During  subsequent  phases,  concepts,  tech¬ 
nological  approaches,  and/or  design  ap¬ 
proaches  (selected  at  the  previous  mile¬ 
stone  decisions)  are  pursued  to  define  the 
program  and  program  risks.  Selected  al¬ 
ternative  concepts  continue  to  be  analyzed, 
and  the  acquisition  strategy,  and  the  vari¬ 
ous  strategies  and  plans  derived  from  it, 
continue  to  be  refined. 

Risk  management  efforts  in  these  phases 
focus  on:  understanding  critical  technol¬ 
ogy,  manufacturing,  and  support  risks, 
along  with  cost,  schedule,  and  perfor¬ 
mance  risks;  and  demonstrating  that  they 
are  being  controlled  before  moving  to  the 
next  milestone.  Note  that  the  accuracy  of 
cost,  schedule,  performance  risk  assess¬ 
ments  should  improve  with  each  succeed¬ 
ing  program  phase  (e.g.,  more  info,  better 
design  documentation,  etc.).  Thus,  par¬ 
ticular  attention  should  be  placed  on  han¬ 
dling  and  monitoring  activities.  Planning 
and  assessment  should  continue  as  new 
information  becomes  available  and  new 
risk  events  are  identified. 

During  these  phases,  the  risk  management 
program  should  be  carried  out  in  an  inte¬ 
grated  Government-contractor  framework 
to  the  extent  possible,  that  allows  the  Gov¬ 
ernment  to  manage  program  risks,  with  the 
contractor  responsible  to  the  PM  for  prod¬ 
uct  and  process  risks  and  for  maintaining 
design  accountability.  Both  the  Govern¬ 
ment  and  contractors  need  to  understand 


the  risks  clearly,  and  jointly  plan  manage¬ 
ment  efforts.  In  any  event,  risk  manage¬ 
ment  needs  to  be  tailored  to  each  program 
and  contract  type. 

3.6  RISK  MANAGEMENT  AND 
MILESTONE  DECISIONS 

Before  a  milestone  review,  the  PM  should 
update  risk  assessments,  explicitly  ad¬ 
dressing  the  risks  in  the  critical  areas,  such 
as  threat,  requirements,  technology,  etc., 
and  identify  areas  of  moderate  or  high 
risk. 

Each  critical  technical  assessment  should 
be  supported  by  subsystems'  risk  assess¬ 
ments,  which  should  be  supported  by  de¬ 
sign  reviews,  test  results,  and  specific 
analyses. 

The  PM  should  present  planned  risk  miti¬ 
gation  actions  for  moderate-  or  high-risk 
areas  at  the  milestone  review  to  determine 
their  adequacy  and  to  ensure  the  efficient 
allocation  of  resources. 

3.7  RISK  MANAGEMENT  AND  THE 
ACQUISITION  STRATEGY 

In  addition  to  providing  the  framework  for 
program  planning  and  execution,  the  ac¬ 
quisition  strategy  serves  several  purposes 
that  are  important  to  risk  management: 

•  Provides  a  master  schedule  for  re¬ 
search,  development,  test,  production,  de¬ 
ployment,  and  critical  events  in  the  acqui¬ 
sition  cycle. 

•  Gives  a  master  checklist  of  the  impor¬ 
tant  issues  and  alternatives  that  must  be 
addressed. 

•  Assists  in  prioritizing  and  integrating 
functional  requirements,  evaluating  alter¬ 
natives,  and  providing  a  coordinated  ap¬ 
proach  to  integrate  diverse  functional 
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issues,  leading  to  the  accomplishment  of 
program  objectives. 

•  Documents  the  assumptions  and 
guidelines  that  led  to  the  initiation  and  di¬ 
rection  of  the  program. 

•  Provides  the  basis  for  the  develop¬ 
ment  and  execution  of  the  various  subor¬ 
dinate  functional  strategies  and  plans. 

The  strategy  structure  should  ensure  a 
sound  program  through  the  management 
of  cost,  schedule,  and  performance  risk.  A 
good  acquisition  strategy  acknowledges 
and  identifies  program  risks  and  forms  the 
basis  for  implementing  a  forward-looking, 
rather  than  reactive,  effective  risk  manage¬ 
ment  effort. 

Acquisition  strategy  should  describe  how 
risk  is  to  be  handled  and  identify  which 
risks  are  to  be  shared  with  the  contractor 
and  which  are  to  be  retained  by 
Government.  The  key  concept  here  is  that 
the  Government  shares  the  risk  with  the 
contractor,  but  does  not  transfer  risk  to  the 
contractor.  The  PMO  always  has  a  respon¬ 
sibility  to  the  system  user  to  develop  a  ca¬ 
pable  system  and  can  never  absolve  itself 
of  that  responsibility.  Therefore,  all  pro¬ 
gram  risks,  whether  primarily  managed 
by  the  PMO  or  by  the  contractor,  must  be 
assessed  and  managed  by  the  PMO. 

Once  the  program  office  has  determined 
how  much  of  each  risk  is  to  be  shared  with 
the  contractor,  it  should  assess  the  total 
risk  assumed  by  the  developing  contrac¬ 
tor  (including  subcontractors).  The  Gov¬ 
ernment  should  not  require  contractors  to 
accept  financial  risks  that  are  inconsistent 
with  their  ability  to  handle  them.  Finan¬ 
cial  risks  are  driven,  in  large  measure,  by 
the  underlying  technical  and  program¬ 
matic  risks  inherent  in  a  program.  The 
Government  contracting  officer  should, 


therefore,  select  the  proper  type  of  con¬ 
tract  based  on  an  appropriate  risk  assess¬ 
ment,  to  ensure  a  clear  relationship 
between  the  selected  contract  type  and 
program  risk.  An  example  would  be  the 
use  of  cost-reimbursable- type  contracts  for 
development  projects. 

3.8  RISK  MANAGEMENT  AND  CAIV 

The  intention  of  CAIV  is  to  establish  bal¬ 
ance  between  cost,  schedule,  performance, 
and  risk  early  in  the  acquisition  process 
and  to  manage  to  a  cost  objective.  CAIV 
requires  that  PMs  establish  aggressive  cost 
objectives,  defined  to  some  degree  by  the 
maximum  level  of  acceptable  risk.  Risks  in 
achieving  both  performance  and  aggres¬ 
sive  cost  goals  must  be  clearly  recognized 
and  actively  managed  through: 

(1 )  continuing  iteration  of  cost/ perfor¬ 
mance/schedule/risk  tradeoffs, 

(2)  identifying  key  performance  and 
manufacturing  process  uncertainties, 
and 

(3)  demonstrating  solutions  before 
production. 

Whereas  DoD  has  traditionally  managed 
performance  risk,  equal  emphasis  must  be 
placed  on  managing  cost  and  schedule 
risks.  An  underlying  premise  of  CAIV  is 
that  if  costs  are  too  great,  and  there  are 
ways  to  reduce  them,  then  the  user  and 
developer  may  reduce  performance  re¬ 
quirements  to  meet  cost  objections.  Cost 
control  and  effective  risk  management  in¬ 
volve  planning  and  scheduling  events  and 
demonstrations  to  verify  solutions  to  cost, 
schedule,  performance  risk  issues. 

User  participation  in  the  trade-off  analysis 
is  essential  to  attain  a  favorable  balance  be¬ 
tween  cost,  schedule,  performance,  and 
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risk.  The  PM  and  user  representatives 
should  identify  risk  and  cost  driving  re¬ 
quirements  during  the  generation  of  the 
ORD  to  know  where  tradeoffs  may  be  pos¬ 
sible.  Risk  assessments  are  critical  to  the 
CAIV  process  since  they  provide  users  and 
developers  with  essential  data  to  assist  in 
the  cost,  schedule,  performance,  risk  trade 
decisions. 

A  cost  for  risk  management  is  directly  re¬ 
lated  to  the  level  of  risk  and  affects  a  pro¬ 
gram  in  two  ways.  First,  costs  are  associ¬ 
ated  with  specific  handling  activities,  for 


example,  a  parallel  development.  Second, 
funds  are  needed  to  cover  the  known  risks 
of  the  selected  system  approach  (i.e., 
funds  to  cover  cost  uncertainty).  PMs  must 
include  the  anticipated  expense  of  man¬ 
aging  risk  in  their  estimates  of  program 
costs.  Decision  makers  must  weigh  these 
costs  against  the  level  of  risk  in  reaching 
program  funding  decisions.  CAIV  re¬ 
quires  that  program  funds  support  the 
level  of  accepted  program  risk  and  that 
risk  management  costs  are  included  in  set¬ 
ting  cost  objectives. 
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4 

RISK  MANAGEMENT  AND 
PROGRAM  MANAGEMENT 


4.1  INTRODUCTION 

Risk  management  as  a  program  manage¬ 
ment  responsibility  can  be  a  comprehen¬ 
sive  and  responsive  management  tool  if  it 
is  properly  organized  and  monitored  at 
the  PM  level.  A  formalized  risk  manage¬ 
ment  program  should  be  well-planned 
and  forward-looking  by  identifying,  ana¬ 
lyzing,  and  resolving  potential  problem 
areas  before  they  occur,  and  by  incorpo¬ 
rating  monitoring  techniques  that  accu¬ 
rately  portray  the  status  of  risks  and  the 
efforts  to  mitigate  them.  Introduction  of 
risk  management  early  in  a  program  em¬ 
phasizes  its  importance  and  encourages 
contractors  and  members  of  the  Govern¬ 
ment  team  to  consider  risk  in  the  daily 
management  functions. 

This  Chapter  addresses  the  relationship  be¬ 
tween  risk  management  and  program  man¬ 
agement  and  suggests  methods  of  intro¬ 
ducing  risk  management  in  a  program,  or¬ 
ganizing  for  risk,  and  training. 

4.2  OVERVIEW 

A  PMO  should  organize  for  risk  manage¬ 
ment,  using  existing  IPTs.  The  PM  may  also 
want  to  use  contractors  to  support  manage¬ 
ment  efforts  or  have  experts  not  involved 
with  the  program  perform  independent 
assessments. 

To  use  risk  management  as  a  program  man¬ 
agement  tool,  the  information  resulting 


from  each  of  the  risk  processes  should  be 
documented  in  a  usable  form  and  available 
to  members  of  the  Government/ industry 
program  team.  This  information  will  pro¬ 
vide  the  basis  for  reporting  risk  and  over¬ 
all  program  information,  both  internally 
and  externally.  Managing  collection  and 
dissemination  of  risk  information  can  be 
enhanced  through  the  use  of  a  Manage¬ 
ment  Information  System  (MIS). 

4.3  PROGRAM  MANAGER  AND 
RISK  MANAGEMENT 

All  PMs  are  responsible  for  establishing 
and  executing  a  risk  management  pro¬ 
gram  that  satisfies  the  policies  contained 
in  DoDD  5000.1.  A  PM  must  balance 
program-unique  requirements  or  circum¬ 
stances  (e.g.,  size  of  the  PMO  staff)  against 
the  demands  of  proven  risk  management 
principles  and  practices.  This  section  ad¬ 
dresses  these  principles  and  practices  and 
provides  a  basis  for  establishing  a  PMO's 
risk  management  organization  and  related 
procedures.  The  following  guidelines  de¬ 
fine  an  approach  to  risk  management. 

4.3.1  Risk  Management  is  a  Program 
Management  Tool 

Risk  management  should  be  integral  to  a 
program's  overall  management.  PMs  must 
take  an  active  role  in  the  process  to  ensure 
that  their  approach  leads  to  a  balanced  use 
of  program  resources,  reflects  their  overall 
management  philosophy,  and  includes 
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Government  and  contractors.  Past  DoD 
practices  have  generally  treated  risk  man¬ 
agement  solely  as  a  system  engineering 
function,  cost-estimating  technique  or 
possibly  as  an  independent  function  dis¬ 
tinct  from  other  program  functions.  To¬ 
day,  risk  management  is  recognized  as  a 
vital  integrated  program  management 
tool  that  cuts  across  the  entire  acquisition 
program,  addressing  and  interrelating 
cost,  schedule,  and  performance  risks. 
The  goal  is  to  make  everyone  involved 
in  a  program  aware  that  risk  should  be  a 
consideration  in  the  design,  develop¬ 
ment,  and  fielding  of  a  system.  It  should 
not  be  treated  as  someone  else's  respon¬ 
sibility.  Specific  functional  areas — such  as 
system  engineering — could  be  charged 
with  implementing  risk  management,  as 
long  as  they  take  the  program  manage¬ 
ment  view  towards  it. 

4.3.2  Risk  Management  is  a 
Formal  Process 

Formal  risk  management  refers  to  a  struc¬ 
tured  process  whereby  risks  are  systemati¬ 
cally  identified,  analyzed,  handled,  and 
monitored.  (A  recommended  structure  is 
described  in  Section  2  of  this  Guide.)  A 
structured  risk  management  process, 
which  is  applied  early,  continuously,  and 
rigorously,  provides  a  disciplined  envi¬ 
ronment  for  decision  making  and  for  the 
efficient  use  of  program  resources. 
Through  a  disciplined  process  PMs  can 
uncover  obscure  and  lower-level  risks  that 
collectively  could  pose  a  major  risk. 

The  need  for  a  formal  risk  management 
process  arises  from  the  nature  of  risk  and 
the  complexity  of  acquisition  programs. 
The  numerous  risks  in  an  acquisition  pro¬ 
gram  are  often  interrelated  and  obscure 
and  change  in  the  course  of  the  develop¬ 
ment  process.  A  formal  approach  is  the 
only  effective  method  to  sort  through 


numerous  risk  events,  to  identify  the  risks 
and  their  interrelationships,  to  pinpoint 
the  truly  critical  ones,  and  to  identify  cost- 
effective  ways  to  reduce  those  risks,  con¬ 
sistent  with  overall  program  objectives. 

A  structured  process  can  reduce  the  com¬ 
plexity  of  an  acquisition  program  by  de¬ 
fining  an  approach  to  assess,  handle, 
monitor,  and  communicate  program  risk. 
The  systematic  identification,  analysis, 
and  mitigation  of  risks  also  offers  a  reli¬ 
able  way  to  ensure  objectivity,  that  is, 
minimize  unwarranted  optimism,  preju¬ 
dice,  ignorance,  or  self-interest.  Further, 
structure  reduces  the  impact  of  personnel 
turnover  and  provides  a  basis  for  train¬ 
ing  and  consistency  among  all  the  func¬ 
tional  areas  of  a  program.  A  structured  risk 
program  may  also  promote  teamwork  and 
understanding  and  improves  the  quality 
of  the  risk  products. 

4.3.3  Risk  Management  is 
Forward-Looking 

Effective  risk  management  is  based  on  the 
premise  that  PMs  must  identify  potential 
problems,  referred  to  as  risk  events,  long 
before  they  can  occur  and  develop  strate¬ 
gies  that  increase  the  likelihood  of  a  fa¬ 
vorable  outcome  to  these  problems.  Ap¬ 
plication  of  this  philosophy  occurs  prima¬ 
rily  by  using  analytical  techniques  that 
give  forward-looking  assessments. 

Typically,  the  early  identification  of  poten¬ 
tial  problems  is  concerned  with  two  types 
of  events.  The  first  are  relevant  to  the  cur¬ 
rent  or  imminent  acquisition  phase  of  a 
program  (intermediate-term),  such  as  sat¬ 
isfying  a  technical  exit  criteria  in  time  for 
the  next  milestone  review.  The  second  are 
concerned  with  the  future  phase(s)  of  a 
program  (long-term)  such  as  potential  risk 
events  related  to  transitioning  a  system 
from  development  to  production. 
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By  analyzing  critical  events,  certain  risks 
can  be  determined.  To  do  this,  one  should 
consider  the  range  of  potential  outcomes 
and  the  factors  that  determine  those  out¬ 
comes.  Through  risk  handling,  a  PM  then 
develops  approaches  that  minimize  risk 
factors.  Paragraph  5.6  of  this  Guide  de¬ 
scribes  some  handling  approaches. 

Choosing  the  proper  risk-handling  op¬ 
tions  requires  that  a  balance  be  struck  be¬ 
tween  the  resources  required  to  imple¬ 
ment  those  options  and  their  payoffs  (both 
intermediate  and  long-term)  and  the  re¬ 
sources  realistically  available. 

4.3.4  Risk  Management  is  Integral  to 
Integrated  Product  and  Process 
Development  (IPPD) 

One  of  the  tenets  of  IPPD  is  multi¬ 
disciplinary  teamwork  through  IPTs, 
which  are  an  integral  part  of  the  defense 
acquisition  oversight  and  review  process. 
The  Integrating  IPT  (IIPT)  is  a  valuable  re¬ 
source  to  assist  in  developing  a  risk  man¬ 
agement  plan  and  should  be  used  accord¬ 
ingly.  The  PM  should  ensure  that  the  re¬ 
quirements  of  the  overarching  IPT  (OIPT) 
are  reflected  in  the  plan. 

Working  with  the  OIPT,  the  PM  can  estab¬ 
lish  the  type  and  frequency  of  risk  manage¬ 
ment  information  that  an  OIPT  requires,  and 
refine  management  organization  and  proce¬ 
dures.  This  should  be  done  during  the  ini¬ 
tial  OIPT  meetings.  OIPTs  will  most  likely 
require  information  concerning: 

•  Known  risks  and  their  characteristics, 
e.g.,  probability  of  occurrence  and  conse¬ 
quences, 

•  Planned  risk-handling  actions, 
funded  and  unfunded, 

•  Achievements  in  controlling  risks  at 
acceptable  levels. 


IIPTs  and  OIPTs  may  also  require  details 
on  the  PM's  risk  management  program, 
access  to  the  risk  management  plan,  and 
the  results  of  specific  risk  assessments.  In 
addition,  PMs  may  want  to  present  se¬ 
lected  information  to  IIPTs  and  OIPTs  to 
help  substantiate  a  position  or  recom¬ 
mendation,  e.g.,  help  support  a  budget 
request. 

4.4  RISK  MANAGEMENT 

ORGANIZATION  IN  THE  PMO 

The  PM,  after  determining  a  preferred  man¬ 
agement  approach,  must  organize  the  pro¬ 
gram  office  and  establish  outside  relation¬ 
ships  in  order  to  manage  risk.  No  particu¬ 
lar  organizational  structure  is  superior; 
however,  experience  provides  some  insights 
into  the  development  of  effective  risk  man¬ 
agement  organizations.  PMs  should  con¬ 
sider  the  following  discussion  in  the  con¬ 
text  of  their  unique  requirements  and  cir¬ 
cumstances  and  apply  those  that  are  suit¬ 
able  to  their  specific  needs. 

4.4.1  Risk  Management 

Organizational  Structure 

A  major  choice  for  each  PM  is  whether  to 
have  a  centralized  or  decentralized  risk  man¬ 
agement  organization.  The  PM  may  choose 
a  centralized  organizational  structure  until 
team  members  become  familiar  with  both  the 
program  and  the  risk  management  process. 
In  a  centralized  approach,  the  PM  estab¬ 
lishes  a  team  that  is  responsible  for  all  as¬ 
pects  of  risk  management.  The  team  would 
write  a  plan,  conduct  assessments,  evalu¬ 
ate  risk-handling  options,  and  monitor 
progress.  Although  this  approach  may  be 
necessary  early  in  a  program,  it  tends  to 
minimize  the  concept  that  risk  manage¬ 
ment  is  a  responsibility  shared  by  all  mem¬ 
bers  of  the  acquisition  team,  whether  Gov¬ 
ernment  or  contractor. 
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Figure  4-1.  Decentralized  Risk  Management  Organization 


The  PM  may  also  choose  to  decentralize. 
The  degree  of  decentralization  depends  on 
the  assignment  of  responsibilities.  Some 
level  of  centralization  is  almost  always  es¬ 
sential  for  prioritizing  risk  across  the  pro¬ 
gram.  A  program  level  integrating  IPT  (See 
Figure  4-1)  or  a  Risk  Management  Board 
may  be  appropriate  for  this  integrating 
function. 

The  decentralized  risk  management  orga¬ 
nization  is  the  most  widely  used  approach, 
which  is  compatible  with  the  DoD's  IPPD 
policy  and  generally  results  in  an  efficient 
use  of  personnel  resources.  In  this  ap¬ 
proach,  risk  management  is  delegated  to 
Program  IPTs. 

The  following  guidelines  apply  to  all  risk 
management  organizations: 

•  The  PM  is  ultimately  responsible  for 
planning,  allocating  resources,  and  execut¬ 
ing  risk  management.  This  requires  the  PM 
to  oversee  and  participate  in  the  risk  man¬ 
agement  process. 


•  The  PM  must  make  optimal  use  of 
available  resources,  i.e.,  personnel,  organi¬ 
zations,  and  funds.  Personnel  and  organi¬ 
zational  resources  include  the  PMO,  func¬ 
tional  support  offices  of  the  host  command, 
the  prime  contractor,  independent  risk  as¬ 
sessors,  and  support  contractors. 

•  Risk  management  is  a  team  function. 
This  stems  from  the  pervasive  nature  of  risk 
and  the  impact  that  risk-handling  plans 
may  have  on  other  program  plans  and  ac¬ 
tions.  In  the  aggregate,  risk  planting,  risk 
assessment,  risk  handling,  and  risk  moni¬ 
toring  affect  all  program  activities  and  or¬ 
ganizations.  Any  attempt  to  implement  an 
aggressive  forward-looking  risk  manage¬ 
ment  program  without  the  involvement  of 
all  PMO  subordinate  organizations  could 
result  in  confusion,  misdirection,  and 
wasted  resources.  The  only  way  to  avoid 
this  is  through  teamwork  among  the  PMO 
organizations  and  the  prime  contractor. 
The  management  organizational  structure 
can  promote  teamwork  by  requiring  strong 
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connectivity  between  that  structure,  the 
various  PMO  organizations,  and  the  prime 
contractor.  The  teams  may  use  independent 
assessments  to  assist  them,  when  required. 

Figure  4-1  portrays  a  decentralized  risk 
management  organization.  This  example 
includes  the  entire  PMO  and  selected  non- 
PMO  organizations,  e.g.,  the  prime  contrac¬ 
tor,  who  are  members  of  the  IPTs.  The  fig¬ 
ure  shows  that  risk  management  is  an  in¬ 
tegral  part  of  program  management  and 
not  an  additional  or  separate  function  to  per¬ 
form.  Hence,  separate  personnel  are  not  des¬ 
ignated  to  manage  risk,  but  rather  all  indi¬ 
viduals  are  required  to  consider  risk  man¬ 
agement  as  a  routine  part  of  their  jobs.  In 
the  figure,  the  risk  coordinator  reports  to 
the  PM,  but  works  in  coordination  with  the 
Program  IPT,  functional  offices,  and  the 
Program  Level  IPT.  As  shown,  this  organi¬ 
zational  structure  is  suited  to  ACAT  I  pro¬ 
grams,  but  PMs  can  tailor  it  to  satisfy  their 
specific  requirements.  The  details  are  depen¬ 
dant  upon  the  contract,  type,  statement  of 
work,  and  other  variable. 

The  organizational  structure  shows  that 
the  PM  is  ultimately  responsible  for  risk 
management.  There  is  a  coordinator  to 
assist  with  this  responsibility  and  act  as 
an  "operations"  officer.  This  maybe  a  full¬ 
time  position  or  an  additional  duty  as  the 
PM  deems  appropriate.  The  coordinator 
should  have  specific  training  and  experi¬ 
ence  in  risk  management  to  increase  the 
chance  of  successful  implementation  and 
to  avoid  common  problems.  A  support 
contractor  may  assist  the  coordinator  by 
performing  administrative  tasks  associ¬ 
ated  with  that  office. 

The  Program  Level  IPT,  composed  of  in¬ 
dividuals  from  the  PMO  and  prime  con¬ 
tractor,  ensures  that  the  PM's  risk  man¬ 
agement  program  is  implemented  and 


program  results  are  synthesized  into  a 
form  suitable  for  decision  making  by  the 
PM  and  OIPT. 

The  inclusion  of  both  Sub-Tier  IPTs  and 
PMO  functional  offices  simply  reflects  that 
not  all  program  management  functions 
will  be  assigned  to  Sub-Tier  IPTs  for 
execution. 

Independent  risk  assessors  are  typically 
hired  when  the  PM  has  specific  cost,  sched¬ 
ule,  performance  concerns  with  a  hardware 
or  software  product  or  engineering  process 
and  wants  an  independent  assessment 
from  an  expert  in  a  particular  field.  The 
duration  of  their  services  is  normally  short, 
and  tailored  to  each  program. 

4.4.2  Risk  Management 
Responsibilities 

This  section  identifies  the  primary  respon¬ 
sibilities  that  could  be  associated  with  a 
decentralized  risk  management  organiza¬ 
tion.  In  assigning  the  responsibilities  to  the 
various  organizational  elements,  the  PM 
should  strike  a  balance  between  a  concen¬ 
tration  of  responsibilities  at  the  higher  lev¬ 
els  and  pushing  them  too  far  down  the  or¬ 
ganizational  structure. 

The  development  of  these  responsibili¬ 
ties,  in  part,  is  based  on  the  premise  that 
risk  management  activities  must  be  spe¬ 
cific — and  assigned  to  individuals,  not 
groups.  The  responsibilities  listed  below 
are  assigned  to  the  leader  of  each  orga¬ 
nizational  element,  recognizing  that  the 
composition  of  each  element  will  be  pro¬ 
gram  unique,  i.e.,  number  of  assigned 
PMO  personnel,  prime  contractor  per¬ 
sonnel,  etc.  The  task  of  further  assigning 
these  responsibilities,  along  with  tailor¬ 
ing  them  to  satisfy  the  needs  and  require¬ 
ments  of  each  program,  remains  for  PMs 
and  their  staffs  to  accomplish. 
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Table  4-1  provides  a  description  of  the  re¬ 
sponsibilities  associated  with  the  decentral¬ 
ized  risk  management  structure,  sorted  by 
notional  organizational  elements  that  may 
make  up  the  risk  management  structure. 

4.5  CONTRACTOR  RISK 
MANAGEMENT 

Experience  has  shown  that  managing  a 
program's  risks  requires  a  close  partner¬ 
ship  between  the  PMO  and  the  prime 
contractor(s).  PMs  must  determine  the  type 
of  support  they  need  from  their  prime 
contractor,  communicate  these  needs 
through  the  Request  for  Proposal  (RFP)  for 
each  acquisition  phase,  and  then  provide 
for  them  in  the  contract.  Preparation  of  the 
RFP  and  source  selection  are  discussed  in 
subsequent  sections. 

4.5.1  Contractor  View  of  Risk 

Contractors  treat  risk  differently  from  the 
Government  because  each  views  risk  from 
a  different  perspective.  The  PM,  in  execut¬ 
ing  his  risk  management  program,  needs 
to  understand  the  contractor  viewpoint. 

Contractors  typically  divide  risks  into  two 
basic  types:  business  risks  and  program 
risks.  Business  risk,  in  the  broadest  sense, 
involves  the  inherent  chance  of  making  a 
profit  or  incurring  a  loss  on  any  given 
contract.  Program  risk  involves,  among 
other  things,  technical,  requirement,  and 
design  uncertainties.  A  contractor's  efforts 
to  minimize  business  risks  may  conflict 
with  a  Government  PM's  efforts  to  lower 
program  risk. 

While  the  government  and  contractors  may 
have  different  views  on  specific  cost,  sched¬ 
ule,  and  performance  risk  levels /ratings, 
they  generally  have  (or  should  have)  simi¬ 
lar  views  of  the  risk  management  process. 
One  exception  may  be  the  requirements 


placed  by  corporate  management — that 
could  conflict  with  the  Government  view 
of  program  risk.  The  similarity,  however, 
does  not  necessarily  lead  to  the  contractor 
having  a  competent  internal  risk  manage¬ 
ment  program.  As  a  Project  Management 
Institute  (PMI)  handbook  points  out,  "On 
most  (contractor)  projects,  responsibility 
for  Project  Risk  is  so  pervasive  that  it  is 
rarely  given  sufficient  central  attention." 
As  a  minimum,  it  is  important  that  the 
PMO  writes  the  RFP  asking  the  contractor 
to  describe  its  risk  management  process, 
including  its  approach  to  managing  any 
specific  areas. 

4.5.2  Government/Contractor 
Relationship 

The  prime  contractor's  support  and  assis¬ 
tance  is  required  even  though  the  ultimate 
responsibility  for  risk  management  rests 
with  the  Government  PM.  Often,  the  con¬ 
tractor  is  better  equipped  to  understand  the 
program  technical  risks  than  the  Govern¬ 
ment  program  office  is.  Both  the  Govern¬ 
ment  and  contractor  need  to  share  infor¬ 
mation,  understand  the  risks,  and  develop 
and  execute  management  efforts.  The  Gov¬ 
ernment  must  involve  the  contractor  early 
in  program  development,  so  that  effective 
risk  assessment  and  reduction  can  occur. 

Therefore,  risk  management  must  be  a  key 
part  of  the  contractor's  management 
scheme.  Although  the  Government  does 
not  dictate  how  the  contractor  should  man¬ 
age  risk,  some  characteristics  of  a  good 
Government/ contractor  relationship 
include: 

•  Clear  definition  of  risks  and  their 
assignment. 

•  Flexibility  for  assignment  of  risks  and 
risk  management  responsibilities  among 
the  teams. 
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Personnel 


Program  Manager 


Risk  Management 
Coordinator 


Program  Level  IPT 

(some  PMOs  use  a 
Risk  Management 
Board  (RMB)  for  this 
responsibility) 


PMO  Sub-Tier  IPTs  & 
Functional  Offices 
(Process)  and  System 
Elements  (Products) 


Independent  Risk 
Assessors 


Table  4-1. 


Job  Responsibility _ _ _ 


Plan,  organize,  direct,  and  control  risk  management. 

Comply  with  DoDD  5000.1 ,  DoD  5000.2-R,  DoDD.4,  and  DoD  5000.4-M 
risk  managements. 

Ensure  that  funds  are  available  to  support  approved  risk-handling  plans. 
Inform  and  advise  MDA,  Cost  Analysis  Improvement  Group  (CAIG)  and 
01 PT  on  program  risk  and  its  mitigation. _ _ _ _ 

Develop  and  maintain  risk  management  plans. 

Provide  risk  management  training. 

Define  the  risk  reporting  scales  to  be  used  by  the  program. 

Develop  and  maintain  a  risk  management  information  system. 

Prepare  risk  management  reports. 

Monitor  compliance  with  DoDD  risk  management  requirements. 

Ensure  that  risk  management  functions  and  tasks  performed  by  the  Sub- 
Tier  IPTs  and  the  PMO  functional  offices  are  fully  integrated  and  in 
compliance  with  assigned  tasks. 

Advise  the  PM  and  Program  Level  IPT  on  the  use  of  risk  management 
sources,  i.e.,  host  command  functional  support  offices,  etc. 

Evaluate  risk  assessments,  risk-handling  plans,  and  risk  monitoring  results 
as  directed  and  recommend  appropriate  actions. 

Advise  the  PM  on  the  use  of  independent  risk  assessors. _ 

Ensure  that  the  risk  management  program  is  implemented,  risk  reduction  is 
accomplished  in  conformance  with  the  PM’s  strategy,  and  the  risk 
management  efforts  of  the  Sub-Tier  IPTs  are  integrated. 

Report  risk  events  to  the  risk  management  coordinator. 

Evaluate  whether  Sub-Tier  IPTs  and  PMO  functional  offices  have  identified 
critical  risks  and  proposed  risk-handling  plans. 

Ensure  that  cost,  schedule,  and  performance  risks  are  compatible. 

Ensure  that  cost,  schedule,  and  performance  risks  are  combined  in  a 
manner  consistent  with  the  plan. _ 


Assess  risks,  recommending  appropriate  risk-handling  strategies  for 
each  identified  moderate  and  high  risk,  developing  and  implementing 
risk-handling  plans,  monitoring  the  results  of  risk-handling  actions,  and 
documenting  all  risk  management  analyses  and  findings  within  the  team’s 
product  area. 

Coordinate  all  risk  management  findings  and  decisions  with  other  Sub-Tier 
IPTs,  PMO  functional  offices,  the  Program  Level  IPT,  and  the  risk- 
management  coordination  office. 

Identify  funding  requirements  to  implement  risk-handling  plans. 

Identify  the  need  for  risk  management  training. 

Report  risk  events  to  the  Program  Level  IPT  and  risk  coordinator. _ 


Perform  independent  Risk  assessment  on  critical  risk  areas  or 
contractor  engineering  processes  that  the  PM  has  specified. 

Report  the  results  of  those  assessments  to  the  PM. 

Work  with  the  risk  management  coordinator. _ _ 


Notional  Description  of  Risk  Management  Responsibilities 
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•  Strong  emphasis  on  best  management 
and  technical  practices  which,  if  followed, 
avoid  unnecessary  risks. 

Regarding  RFP  development,  discussed 
later  in  this  section,  information  is  pro¬ 
vided  on  how  these  characteristics  should 
be  addressed. 

The  Government/ contractor  partnership 
can  be  forged  in  at  least  two  ways.  First, 
the  PMO  should  include  the  prime 
contractor(s)  in  the  top-level  risk  planning 
and  assessment  activities.  This  includes 
understanding  and  factoring  in  such  issues 
as  user  requirements,  affordability  con¬ 
straints,  and  schedule  limitations.  Second, 
the  PMO  should  include  in  advance  spe¬ 
cific  risk  assessment  and  handling  tasks  as 
key  contractual  efforts  during  the  concept 
exploration  and  program  definition  and 
risk  reduction  phases. 

Forming  a  joint  Government/contractor 
evaluation  team  is  a  good  way  of  fostering 
an  effective  partnership.  This  is  especially 
true  in  a  program's  early  stages  when  un¬ 
certainty  is  high  and  both  parties  must  fre¬ 
quently  assess  risks.  These  assessments, 
properly  handled,  involve  multidisciplinary 
efforts  requiring  subject-matter  experts  from 
both  the  prime  contractor  and  Government. 
This  joint  team  should  evaluate  the  pro¬ 
posed  program  in  detail  and  explore  the  in¬ 
herent  program  risks,  the  proposed  han¬ 
dling  strategies,  the  detailed  development 
schedule,  and  the  contractor's  developmen¬ 
tal  resources  (people,  facilities,  processes, 
tools,  etc.). 

A  management  approach  using  multiple 
teams  is  the  best  approach  to  use,  e.g., 
Sub-Tier  IPTs.  Joint  team(s)  should  be  es¬ 
tablished  at  the  beginning  of  each  de¬ 
velopment  phase  to  assess  the  risks  to  be 
overcome  in  that  phase  and  to  determine 


the  handling  technique(s)  to  be  used. 
Requirements  for  contractor  participation 
on  the  team(s)  should  be  identified  in  the 
RFP  and  subsequent  contract. 

4.6  RISK  MANAGEMENT  AND  THE 
CONTRACTUAL  PROCESS 

4.6.1  Risk  Management: 

Pre-Contract  Award 

The  contractor's  developmental  and  manu¬ 
facturing  processes  and  tools,  the  availabil¬ 
ity  and  skill  of  personnel,  and  the  previ¬ 
ous  experience  of  the  Government  and  con¬ 
tractor  team  all  influence  their  ability  to 
handle  the  proposed  system  development 
and  production.  Therefore,  an  effective  risk 
management  process  includes  an  evalua¬ 
tion  of  the  capabilities  of  the  potential 
contractors. 

4.6.2  Early  Industry  Involvement: 
Industrial  Capabilities  Review 

An  Industrial  Capabilities  Review  is  a 
powerful  tool  available  to  PMs  for  deter¬ 
mining  general  industrial  capabilities.  To 
avoid  potential  problems  in  the  subse¬ 
quent  competitive  process  and  to  ensure 
that  a  "level  playing  field"  is  maintained, 
an  announcement  in  the  Commerce  Busi¬ 
ness  Daily  should  be  made  to  inform  all 
potential  offerors  that  the  Government 
plans  to  conduct  an  Industrial  Capabili¬ 
ties  Review  and  to  request  responses  from 
all  interested  parties.  Below  is  a  general 
approach  that  PMOs  may  find  readily 
adaptable  to  any  type  of  capability  review. 
The  basic  steps  in  the  process  are  to: 

•  Obtain  the  Source  Selection  Authority's 
approval  to  conduct  the  review. 

•  Establish  the  criteria  for  the  capability. 

•  Identify  the  potential  contractors  who 
will  participate  in  the  review. 
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•  Provide  an  advance  copy  of  the  review 
material  to  those  contractors. 

•  Select  the  review  team,  ensuring  that 
it  has  the  necessary  mix  of  talent. 

•  Train  the  team  on  the  purpose  of  the 
review  and  review  criteria. 

•  Conduct  the  review  and  evaluate  the 
results. 

•  Provide  feedback  to  each  contractor 
on  the  results  of  their  review  and  assess¬ 
ment. 

•  Provide  the  results  to  the  PM. 

This  review  is  an  appraisal  of  general  in¬ 
dustrial  capabilities  and  supports  identi¬ 
fying  potential  program  risks  and  best 
practices  rather  than  evaluating  specific 
contractors. 

Regardless  of  the  approach,  the  PMO 
should  determine  what  specific  informa¬ 
tion  is  needed.  DoD  4245. 7-M  is  a  good 
guide  to  help  tailor  a  set  of  questions  for 
the  contractors.  The  questions  generally 
focus  on  two  areas  consistent  with  protec¬ 
tion  of  contractor  proprietary  information. 

•  What  is  the  state-of-the-art  of  the  tech¬ 
nology  proposed  for  use  in  the  system? 

•  What  are  the  general  developmental/ 
manufacturing  capabilities  of  the  potential 
contractors  (including  experience,  tools, 
processes,  etc.)  as  compared  to  industry 
best  practices? 

Table  4-2  shows  some  of  the  specific  areas 
or  sources  for  risk  identification.  It  includes 
a  number  of  areas  (threat,  requirements,  de¬ 
sign,  etc.)  that  have  been  shown  through 
experience  to  contain  risk  events  that  tend 
to  be  more  critical  than  others,  and  which 
ones  should  receive  the  most  management 


attention.  Risk  events  are  determined  by  ex¬ 
amining  WBS  element  product  and  pro¬ 
cesses  in  terms  of  risk  areas.  Process  areas 
are  specifically  addressed  in  Dot)  4245.7M. 
They  are  general  in  that  areas  of  risk  could 
be  present  in  any  program  from  either 
source  (WBS  or  process).  They  are  intended 
as  a  list  of  "top-level"  risk  sources  that  will 
focus  attention  on  a  specific  area.  The  PMO 
and  contractor(s)  will  have  to  examine  lower 
levels  to  understand  the  actual  risks  that  are 
present  in  their  program  and  to  develop  an 
effective  management  plan.  The  risks  shown 
are  not  intended  to  serve  as  a  simple  check¬ 
list  that  one  should  apply  directly,  then  con¬ 
sider  the  program  risk-free  if  none  of  the 
listed  risks  are  present. 

An  examination  of  the  program  in  these  ar¬ 
eas  can  help  to  develop  the  final  program 
acquisition  strategy  and  the  risk-sharing 
structure  between  the  Government  and  in¬ 
dustry.  The  PMO  can  also  use  the  results 
to  adjust  the  RFP  for  the  next  phase  of  the 
program. 

4.6.3  Developing  the  Request 
for  Proposal 

The  RFP  should  communicate  to  all 
offerors  the  concept  that  risk  management 
is  an  essential  part  of  the  Government's 
acquisition  strategy. 

Before  the  draft  RFP  is  developed  using  the 
results  of  the  Industrial  Capabilities  Re¬ 
view,  the  PMO  should  conduct  a  risk  as¬ 
sessment  to  ensure  that  the  program  de¬ 
scribed  in  the  RFP  is  executable  within  the 
technical,  schedule,  and  budget  con¬ 
straints.  Based  on  this  assessment,  a  pro¬ 
gram  plan,  an  integrated  master  schedule, 
and  life-cycle  cost  (LCC)  estimate  may  be 
prepared.  The  technical,  schedule,  and 
cost  issues  should  be  discussed  in  the  pre¬ 
proposal  conference(s)  before  the  draft 
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Risk  Area 

Significant  Risks 

Threat 

Uncertainty  in  threat  accuracy. 

Sensitivity  of  design  and  technology  to  threat. 

Vulnerability  of  system  to  threat  and  threat  countermeasures. 

Vulnerability  of  program  to  intelligence  penetration. 

Requirements 

Operational  requirements  not  properly  established  or  vaguely  stated. 
Requirements  are  not  stable. 

Required  operating  environment  not  described. 

Requirements  do  not  address  logistics  and  suitability. 

Requirements  are  too  constrictive — identify  specific  solutions  that  force 
high  cost. 

Design 

Design  implications  not  sufficiently  considered  in  concept  exploration. 

System  will  not  satisfy  user  requirements. 

Mismatch  of  user  manpower  or  skill  profiles  with  system  design  solution  or 
human-machine  interface  problems. 

Increased  skills  or  more  training  requirements  identified  late  in  the 
acquisition  process. 

Design  not  cost  effective. 

Design  relies  on  immature  technologies  or  “exotic”  materials  to  achieve 
performance  objectives. 

Software  design,  coding,  and  testing. 

Test  and  Evaluation 

Test  planning  not  initiated  early  in  program  (Phase  0). 

Testing  does  not  address  the  ultimate  operating  environment. 

Test  procedures  do  not  address  all  major  performance  and  suitability 
specifications. 

Test  facilities  not  available  to  accomplish  specific  tests,  especially  system- 
level  tests. 

Insufficient  time  to  test  thoroughly. 

Simulation 

Same  risks  as  contained  in  the  Significant  Risks  for  Test  and  Evaluation. 

M&S  are  not  verified,  validated,  or  accredited  for  the  intended  purpose. 
Program  lacks  proper  tools  and  modeling  and  simulation  capability  to 
assess  alternatives. 

Technology 

Program  depends  on  unproved  technology  for  success — there  are  no 
alternatives. 

Program  success  depends  on  achieving  advances  in  state-of-the-art 
technology. 

Potential  advances  in  technology  will  result  in  less  than  optimal  cost- 
effective  system  or  make  system  components  obsolete. 

Technology  has  not  been  demonstrated  in  required  operating  environment. 
Technology  relies  on  complex  hardware,  software,  or  integration  design. 

Logistics 

Inadequate  supportability  late  in  development  or  after  fielding,  resulting  in 
need  for  engineering  changes,  increased  costs,  and/or  schedule  delays. 

Life-cycle  costs  not  accurate  because  of  poor  logistics  supportability 
analyses. 

Logistics  analyses  results  not  included  in  cost-performance  tradeoffs. 

Design  trade  studies  do  not  include  supportability  considerations. 

Table  4-2.  Significant  Risks  by  Critical  Risk  Areas 
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Risk  Area 

Significant  Risks 

Production/ 

Facilities 

Production  implications  not  considered  during  concept  exploration. 

Production  not  sufficiently  considered  during  design. 

Inadequate  planning  for  long  lead  items  and  vendor  support. 

Production  processes  not  proven. 

Prime  contractors  do  not  have  adequate  plans  for  managing 
subcontractors. 

Sufficient  facilities  not  readily  available  for  cost-effective  production. 

Contract  offers  no  incentive  to  modernize  facilities  or  reduce  cost. 

Concurrency 

Immature  or  unproven  technologies  will  not  be  adequately  developed 
before  production. 

Production  funding  will  be  available  too  early — before  development  ettort 
has  sufficiently  matured. 

Concurrency  established  without  clear  understanding  of  risks. 

Capability  of 

Developer 

Developer  has  limited  experience  in  specific  type  of  development. 

Contractor  has  poor  track  record  relative  to  costs  and  schedule. 

Contractor  experiences  loss  of  key  personnel. 

Prime  contractor  relies  excessively  on  subcontractors  for  major 
development  efforts. 

Contractor  will  require  significant  capitalization  to  meet  program 
requirements. 

Cost/Funding 

Realistic  cost  objectives  not  established  early. 

Marginal  performance  capabilities  incorporated  at  excessive  costs- 
satisfactory  cost-performance  tradeoffs  not  done. 

Excessive  life-cycle  costs  due  to  inadequate  treatment  of  support 
requirements. 

Significant  reliance  on  software. 

Funding  profile  does  not  match  acquisition  strategy. 

Fundinq  profile  not  stable  from  budget  cycle  to  budget  cycle. 

Schedule 

Schedule  not  considered  in  trade-off  studies. 

Schedule  does  not  reflect  realistic  acquisition  planning. 

APB  schedule  objectives  not  realistic  and  attainable. 

Resources  not  available  to  meet  schedule. 

Management 

Acquisition  strategy  does  not  give  adequate  consideration  to  various 
essential  elements,  e.g.,  mission  need,  test  and  evaluation,  technology,  etc. 

Subordinate  strategies  and  plans  are  not  developed  in  a  timely  manner  or 
based  on  the  acquisition  strategy. 

Proper  mix  (experience,  skills,  stability)  of  people  not  assigned  to  PMO  or 
to  contractor  team. 

Effective  risk  assessments  not  performed  or  results  not  understood  and 
acted  upon. 

Table  4-2.  Significant  Risks  by  Critical  Risk  Areas 
(Continued) 
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RFP  is  released.  In  this  way,  critical  risks 
inherent  in  the  program  can  be  identified 
and  addressed  in  the  RFP.  In  addition,  this 
helps  to  establish  key  risk-management 
contractual  conditions.  The  RFP  should 
encourage  offerors  to  extend  the  contract 
WBS  (CWBS)  to  reflect  how  they  will  iden¬ 
tify  all  elements  at  any  level  that  are  ex¬ 
pected  to  be  high  cost  or  high  risk.  The 
RFP  should  also  encourage  offerors  to  cite 
any  elements  of  the  CWBS  provided  in  the 
draft  RFP  that  are  not  consistent  with  their 
planned  approach. 

In  the  solicitation,  PMs  may  ask  offerors  to 
include  a  risk  analysis  and  a  description  of 
their  management  plans,  and  also  to  develop 
a  supporting  program  plan  and  an  integrated 
master  schedule  in  their  proposals.  These 
proposals  will  support  the  Government's 
source  selection  evaluation  and  the  formula¬ 
tion  of  a  most  probable  cost  estimate  for  each 
proposal.  In  addition,  the  RFP  may  identify 
the  requirement  for  periodic  risk  assessment 
reports  that  would  serve  as  inputs  to  the  PM's 
assessment  and  monitoring  processes 
thereby  ensuring  that  risks  are  continuously 
assessed. 

4.6.4  The  Offeror's  Proposal 

The  offerors  should  develop  the  proposed 
program  plans  and  documentation  at  a 
level  that  is  adequate  to  identify  risks, 
develop  associated  management  activities 
that  they  will  use  throughout  the  program, 
and  integrate  resources,  technical  perfor¬ 
mance  measures,  and  schedule  in  the  pro¬ 
posed  program  plans.  Program  plans 
should  extend  the  CWBS  to  reflect  the 
offeror's  approach  and  include  the  sup¬ 
porting  activities,  critical  tasks,  and  pro¬ 
cesses  in  the  CWBS  dictionary.  The  associ¬ 
ated  schedules  for  each  should  be  incor¬ 
porated  into  an  integrated  master  sched¬ 
ule.  Plans  should  also  have  an  estimate  of 


the  funds  required  to  execute  the  program 
and  include  a  breakout  of  resource  require¬ 
ments  for  high-risk  areas. 

The  information  required  and  the  level  of 
detail  will  depend  on  the  acquisition  phase, 
the  category,  and  criticality  of  the  program, 
as  well  as  on  the  contract  type  and  value. 
However,  the  detail  submitted  with  the 
proposal  must  be  at  a  sufficiently  low  level 
to  allow  identification  of  possible  conflicts 
in  the  planned  acquisition  approach  and 
to  support  the  Government's  proposal 
evaluation.  Generally,  the  CWBS  should  be 
defined  below  level  3,  by  the  contractor, 
only  to  the  extent  necessary  to  capture  those 
lower  level  elements  that  are  high  cost,  high 
risk,  or  of  high  management  interest. 

4.6.5  Basis  for  Selection 

DoD  acquisition  management  must  focus  on 
balancing  cost,  schedule,  performance,  and 
risk  by  selecting  the  contractor  team  that  pro¬ 
vides  the  best  value  to  the  user  within  ac¬ 
ceptable  risk  limits.  Therefore,  the  RFP/ 
Source  Selection  process  must  evaluate  each 
offeror's  capability  for  meeting  product  and 
process  technical,  cost  and  schedule  require¬ 
ments  while  addressing  and  controlling  the 
risks  inherent  in  a  program. 

The  evaluation  team  should  discriminate 
among  offerors  based  upon  the  following: 

•  Risks  determined  by  comparison  with 
the  best  practices  baseline. 

•  Ability  to  perform  with  a  focus  on  the 
critical  risk  elements  inherent  in  the 
program. 

•  Adherence  to  requirements  associated 
with  any  mandatory  legal  items. 

•  Past  performance  on  efforts  similar  to 
the  proposed  program  being  evaluated. 
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The  process  of  choosing  among  offerors  may 
be  enhanced  if  the  evaluation  team  includes 
risk  management  as  a  "source  selection  dis¬ 
criminator."  Risk  management  then  becomes 
an  important  factor  in  the  Source  Selection 
Authority  determination  of  who  provides  the 
most  executable  program. 

4.6.6  Source  Selection 

The  purpose  of  a  source  selection  is  to  select 
the  contractor  whose  cost,  schedule  and  per¬ 
formance  can  best  be  expected  to  meet  the 
Government's  requirements  at  an  affordable 
price.  To  perform  this  evaluation,  the  Gov¬ 
ernment  must  assess  both  proposal  risk  and 
performance  risk  for  each  proposal.  These  risk 
assessments  must  be  done  entirely  within  the 
boundaries  of  the  source  selection  process. 
Previous  assessments  of  any  of  the  offerors 
may  not  be  applicable  or  allowable. 

4.6.6.1  Proposal  Risk.  This  refers  to  the 
risk  associated  with  the  offeror's  proposed 
approach  to  meet  the  Government  cost, 
schedule,  and  performance  requirements. 
The  evaluation  of  proposal  risk  includes 
an  assessment  of  proposed  time  and  re¬ 
sources  and  recommended  adjustments. 
This  assessment  should  be  performed  ac¬ 
cording  to  the  definitions  and  evaluation 
standards  developed  for  the  source  selec¬ 
tion.  Proposal  risk  is,  in  essence,  a  moder¬ 
ate  expansion  of  past  evaluation  processes. 
Historically,  evaluators  selected  contractors 
who  demonstrated  that  they  understood 
the  requirements  and  offered  the  best  value 
approach  to  meeting  the  Government's 
needs.  The  expansion  on  this  concept  is  the 
specific  consideration  of  risk. 

Technical  and  schedule  assessments  are 
primary  inputs  to  the  most  probable  cost 
estimate  for  each  proposal.  It  is  important 
to  estimate  the  additional  resources  needed 
to  control  any  risks  that  have  moderate  or 


high  risk  ratings.  Offerors  may  define  them 
in  terms  of  additional  time,  personnel  load¬ 
ing,  hardware,  or  special  actions  such  as 
additional  tests.  However,  whatever  the 
type  of  the  required  resources,  it  is  essen¬ 
tial  that  cost  estimates  be  integrated  and 
consistent  with  the  technical  and  schedule 
evaluations. 

4.6.6.2  Performance  Risk.  A  performance 
risk  assessment  is  an  evaluation  of  the 
contractor's  past  and  present  performance 
record  to  establish  a  level  of  confidence  in 
the  contractor's  ability  to  perform  the  pro¬ 
posed  effort. 

A  range  of  methods  are  available  to  the  PM 
to  evaluate  performance  risk.  The  Perfor¬ 
mance  Risk  Assessment  Group  (PRAG)  is  a 
group  of  experienced  Government  person¬ 
nel  that  are  appointed  by  the  source  selec¬ 
tion  advisory  council  Chairperson  to  permit 
performance  risk  to  be  used,  if  appropriate. 
Performance  risk  may  be  separately  assessed 
for  each  evaluation  factor  or  as  a  whole  with 
the  assessment  provided  directly  to  the 
source  selection  advisory  council/ authority 
for  final  decision  or  indirectly  through  the 
Source  Selection  Evaluation  Board.  The 
assessment  relies  heavily  (although  not  ex¬ 
clusively)  on  the  contractor  performance 
evaluations  and  surveys  submitted  by  the 
PMO  and  Defense  Contract  Management 
Command  (DCMC). 

4.7  RISK  MANAGEMENT: 

POST-CONTRACT  AWARD 

Post-contract  award  risk  management 
builds  on  the  work  done  during  the  pre¬ 
contract  award  phase.  With  the  award  of 
the  contract,  the  relationship  between  the 
Government  and  the  contractor  changes  as 
teams  are  formed  to  address  program  risk. 
These  teams  should  validate  pre-contract 
award  management  plans  by  reviewing 
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assessments,  handling  plans,  and  monitor¬ 
ing  intentions.  The  extent  of  assessments 
increases  as  the  contractor  develops  and 
refines  his  design,  test  and  evaluation,  and 
manufacturing  plans.  The  Government 
PMO  should  work  with  the  contractor  to 
refine  handling  plans. 

The  process  begins  with  an  Integrated 
Baseline  Review  (IBR)  after  contract  award 
to  ensure  that  reliable  plans  and  perfor¬ 
mance  measurement  baselines  capture  the 
entire  scope  of  work,  are  consistent  with 
contract  schedule  requirements,  and  have 
adequate  resources  assigned  to  complete 
program  tasks.  The  IBR  could  be  conducted 
to  incorporate  other  steps  identified  below. 
These  steps  suggest  an  approach  that  the 
PMO  might  take  to  initiate  the  program's 
risk  management  plans  and  activities  after 
contract  award.  They  are  intended  to  be  a 
starting  point,  and  the  PMO  should  tailor 
the  plan  to  reflect  each  program's  unique 
needs. 

•  Conduct  initial  meeting  with  the  con¬ 
tractor  to  describe  the  program's  objectives 
and  approach  to  managing  risks.  The  PM 
may  also  present  the  risk  management 
plan. 

•  Train  members  of  PMO  and  con¬ 
tractor's  organization  on  risk  management 
basics,  incorporating  the  program's  man¬ 
agement  plan  and  procedures  into  the 
training. 

•  Review  the  pre-contract  award  risk 
plan  with  the  PMO  and  contractor,  revise 
it  as  necessary,  and  share  results  with  the 
contractor. 

•  Conduct  in-depth  review  of  the  pre¬ 
contract  award  risk  assessments  and  ex¬ 
pand  the  review  to  include  any  new  in¬ 
formation  obtained  since  the  award  of  the 
contract. 


•  Review  and  revise  risk-handling 
plans  to  reflect  the  reassessment  of  risks. 

•  Review  the  program's  documentation 
requirements  with  the  contractor.  Ensure 
that  the  PMO  and  contractor  understand 
the  purpose,  format,  and  contents  of  vari¬ 
ous  risk  reports. 

•  Initially,  it  may  be  necessary  to  estab¬ 
lish  a  formalized  PMO-contractor  risk 
management  organization  for  the  program, 
consistent  with  the  terms  of  the  contract. 

•  Working  with  the  contractor,  refine 
the  risk-monitoring  plans  and  procedures. 

•  Establish  the  program  reporting  re¬ 
quirements  with  the  contractor.  Describe 
the  risk  management  information  system 
that  the  program  has  established,  includ¬ 
ing  procedures  for  providing  information 
for  data  entry,  and  identify  reports  for  the 
PMO  and  contractor. 

•  In  conjunction  with  the  contractor, 
identify  other  risk-management  activities 
that  need  to  be  performed. 

•  Manage  the  program  risk  in  accor¬ 
dance  with  the  risk  management  plan  and 
contract. 

•  Working  with  the  contractor,  refine 
the  risk-monitoring  plans  and  procedures 
and  develop  appropriate  measures  and 
metrics  to  track  moderate-  and  high-risk 
items. 

4.8  RISK  MANAGEMENT 
REPORTING  AND 
INFORMATION  SYSTEM 

The  PMO  should  have  a  practical  method 
for  risk-management  reporting,  and  an  in¬ 
formation  system  that  supports  a  risk  man¬ 
agement  program.  The  reporting  needs  of 
the  PM  establish  the  type,  format,  and  fre- 
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quency  of  information  sharing.  The  IPT  con¬ 
cept  suggests  that  the  entire  acquisition  pro¬ 
gram  team  needs  access  to  the  risk  manage¬ 
ment  information,  and  the  prime 
contractor(s)  should  have  access  to  informa¬ 
tion,  consistent  with  acquisition  regulations. 
The  reporting  and  information  system  cho¬ 
sen  may  be  Government-  or  contractor- 
owned.  See  Chapter  5  for  an  example  of  an 
MIS. 

4.9  RISK  MANAGEMENT  TRAINING 

A  successful  management  program  de¬ 
pends,  to  a  large  extent,  on  the  level  of  risk 
management  training  the  PMO  members 
and  the  functional  area  experts  receive.  The 
training  will  prepare  them  for  critical  tasks, 
such  as  risk  assessments.  DoD  schools  of¬ 
fer  some  risk-management  training;  how¬ 
ever,  PMs  will  need  to  organize  and  con¬ 
duct  principal  training  for  the  program  of¬ 
fice.  A  three-part  framework  for  training 
covers  program-specific  risk  management 
issues,  general  structure  and  process,  and 
techniques. 

(1)  The  program-specific  training 
should  ensure  that  everyone  has  a  common 
vision.  It  should  cover  the  acquisition 
strategy,  the  companion  risk  management 
plan,  the  PM's  risk-management  structure 
and  associated  responsibilities,  and  the 
MIS. 


(2)  The  following  topics  provide  a  start¬ 
ing  point  for  general  training  syllabus  de¬ 
velopment.  The  final  syllabus  should  be 
tailored  to  meet  the  program's  specific 
needs.  Table  4-3  provides  a  list  of  references 
that  will  be  useful  in  developing  the  sylla¬ 
bus  and  lesson  plans. 

•  Concept  of  Risk 

•  Risk  Planning 

•  Risk  Identification 

•  Risk  Analysis  (as  applicable) 

•  Risk  Handling 

•  Risk  Monitoring. 

(3)  The  third  area  of  training  concerns 
risk-management  techniques,  concentrating 
on  the  techniques  the  PMO  plans  to  employ. 
The  training  should  focus  on  how  to  use  the 
techniques  and  should  include  examples  of 
their  use.  Chapter  5,  Risk  Management  Tech¬ 
niques,  of  this  Guide  provides  a  starting  point. 
It  contains  a  general  discussion  of  a  set  of 
techniques  that  address  all  elements  of  the 
risk  management  process.  The  discussion  of 
each  technique  contains  a  list  of  references 
that  provide  a  more  in-depth  description  of 
the  technique.  The  set  of  techniques  is  not 
exhaustive  and  the  program  office  should 
add  to  the  list,  if  necessary. 
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Document 


Description 


DoD  4245.7-M,  Transition  from  Development  Provides  a  structure  for  identifying  technical  risk 
to  Production,  September  1 985.  areas  in  the  transition  from  a  program’s  development 

to  production  phases.  The  structure  is  geared  toward 
development  programs  but,  with  modifications,  could 
be  used  for  any  acquisition  program.  The  structure 
identifies  a  series  of  templates  for  each  of  the 
development  contractor’s  critical  engineering 
processes.  The  template  includes  potential  areas  of 
risk  and  methods  for  reducing  risk  in  each  area. 

Risk  Management  Concepts  and  Guidance,  Devoted  to  various  aspects  of  risk  management. 
Defense  Systems  Management  College, 

March  1 989.  (Superseded  by  this  Risk 

Management  Guide.) _ _ '  _ _ _ 

Systems  Engineering  Management  Guide,  Devoted  to  risk  analysis  and  management  and 

Defense  Systems  Management  College,  provides  a  good  overview  of  the  risk  management 

January  1 990,  Section  15. _ process. _ _ _ _ 

Continuous  Risk  Management  Guide,  Provides  a  risk  management  methodology  similar  to 

Software  Engineering  Institute,  Carnegie  the  one  described  in  the  Deskbook.  Its  value  is  that  is 
Mellon  University,  1 996.  subdivides  each  process  into  a  series  of  steps;  this 

provides  useful  insights.  Appendix  A  describes  40 
risk-management  techniques,  the  majority  of  which 
are  standard  management  techniques  adapted  to  risk 
management.  This  makes  them  a  useful  supplement 
to  the  Deskbook  identified  techniques. _ . 

A  Systems  Engineering  Capability  Maturity  Describes  one  approach  to  conducting  an  Industry 
Model,  Version  1 .0  Software  Engineering  Capabilities  Review.  Section  PA  1 0  (pp.  4-72-4-76) 
Institute  (Carnegie  Mellon  University),  discusses  software  risk  management.  The  material 

Handbook  SECMM-94-04,  December  1 994.  presented  in  this  handbook  also  can  be  tailored  to 

apply  to  system  and  hardware  risk. _ 

A  Systems  Engineering  Capability  Maturity  Describes  an  approach  to  assess  the  software 
Model,  Version  1 .01  Software  Engineering  acquisition  processes  of  the  acquiring  organization 
Institute  (Carnegie  Mellon  University),  and  identifies  areas  for  improvement. 

Technical  Report,  December  1996. _ _ _ 

Capability  Maturity  Model  for  Software  (SM-  This  is  a  tool  that  allows  an  acquiring  organization  to 
CMM),  Version  1 .1  ./CMU/SEI-93-TR-24,  assess  the  software  capability  maturity  of  an 

February  1993. _ organization.  _ 

Taxonomy-Based  Risk  Identification,  Describes  a  method  for  facilitating  the  systematic  and 

Software  Engineering  Institute,  Carnegie  repeatable  identification  of  risks  associated  with  the 
Mellon  University,  CMU/SEI-93-TR-6  (ESC-  development  of  a  software-intensive  project.  This 
TR-93-1 83,  June  1 993.  method  has  been  tested  in  active  Government- 

funded  defense  and  civilian  software  development 
projects.  The  report  includes  macro-level  lessons 
learned  from  the  field  tests. _ 

NAVSO  P-6071.  Navy  “best  practices”  document  with  recommended 

implementations  and  further  discussion  on  the 

_  material  in  DoD  4245.7-M.  _ 

Risk  Management,  AFMC  Pamphlet  63-101,  An  excellent  pamphlet  on  risk  management  that  is 
July  1 997.  intended  to  provide  PMs  and  the  PMO  with  a  basic 

understanding  of  the  terms,  definitions,  and 
processes  associated  with  effective  risk 
management.  It  is  very  strong  on  how  to  perform  pre- 
contract  award  risk  management. _ 

Table  4-3.  Risk  Management  Reference  Documents 
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Document 

Description 

Defense  Acquisition  Deskbook 

Primary  reference  tool  for  defense  acquisition  work 
force;  contains  over  1 ,000  mandatory  and 
discretionary  publications  and  documents  which 
promulgate  acquisition  policy  and  guidance,  (http:// 
www.deskbook.osd.mil) 

Acquisition  Software  Development 

Capability  Evaluation,  AFMC  Pamphlet  63- 
103, 15  June  94. 

Describes  one  approach  to  conducting  an  Industry 
Capabilities  Review.  This  two-volume  pamphlet  was 
generated  from  material  originated  at  Aeronautical 
Systems  Center.  The  concepts  support  evaluations 
during  source  selection  and  when  requested  by  IPTs. 
The  material  presented  in  this  pamphlet  also  can  be 
tailored  to  apply  to  system  and  hardware  risk 
management. 

Risk  Management  Critical  Process 
Assessment  Tool,  Air  Force  SMC/AXD, 

Version  2,  9  June  1998. 

Provides  guidance  and  extensive  examples  for 
developing  RFP  Sections  “L”  and  “M,”  plus  source 
selection  standards  or  risk  management.  Also 
includes  technical  evaluation  and  review  questions, 
which  are  helpful  for  assessing  a  risk  management 
process;  and  risk  trigger  questions,  which  are  helpful 
for  risk  identification. 

NAVSO  P-3686,  Top  Eleven  Ways  to 

Manage  Technical  Risk,  October  1998. 

Contains  Navy  approach  to  risk  management  with 
baseline  information,  explanations,  and  best 
practices  that  contribute  to  a  well-founded  technical 
risk  management  program. 

Table  4-3.  Risk  Management  Reference  Documents 
(Continued) 
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5 

RISK  MANAGEMENT  TECHNIQUES 


5.1  INTRODUCTION 

This  Chapter  provides  top-level  informa¬ 
tion  on  a  number  of  techniques  currently 
used  in  DoD,  and  a  combination  of  tech¬ 
niques  Used  by  the  Services,  industry,  and 
academia.  Collectively,  they  focus  on  the 
components  of  the  risk  management  pro¬ 
cess  and  address  critical  risk  areas  and  pro¬ 
cesses.  The  write-ups  describe  the  tech¬ 
niques  and  give  information  on  their  ap¬ 
plication  and  utility.  The  descriptions  are 
at  a  level  of  detail  that  should  permit  po¬ 
tential  users  to  evaluate  the  suitability  of 
the  techniques  for  addressing  their  needs; 
however,  the  material  does  not,  in  most 
cases,  provide  all  the  information  that  is 
required  to  use  a  technique.  Readers  will 
find  that  if  a  particular  technique  looks 
promising,  they  can  obtain  enough  infor¬ 
mation  from  the  references  and  tools  that 
will  enable  program  offices  to  apply  them. 
The  descriptions  are  in  a  format  that  aids 
comparison  with  other  approaches. 

5.2  OVERVIEW 

Techniques  are  available  to  support  risk 
management  activities.  None  are  required 
by  DoD,  but  some  have  been  successfully 
used  in  the  past  by  DoD  PMs.  Many  of 
the  techniques  support  processes  that  are 
part  of  sound  management  and  systems 
engineering  and  give  Government  and 
contractor  PMs  the  tools  for  considering 
risk  when  making  decisions  on  managing 
the  program. 


Several  tools  have  been  developed  to  sup¬ 
port  each  of  the  components  of  the  risk 
management  process,  i.e.,  planning,  assess¬ 
ing,  handling,  and  monitoring  and  docu¬ 
menting.  Although  tool  developers  may 
claim  otherwise,  none  are  integrated  to  to¬ 
tally  satisfy  all  needs  of  a  PM.  Most  likely, 
a  PM  will  choose  an  overall  risk  strategy, 
write  a  plan  to  reflect  his  strategy,  review 
the  list  of  proven  techniques  to  support  the 
components  of  risk  management,  assess  the 
techniques  against  the  program's  needs 
and  available  resources,  tailor  the  tech¬ 
niques  to  suit  the  needs  of  the  program, 
and  train  program  office  members  to 
implement  the  plan. 

5.3  RISK  PLANNING  TECHNIQUES 

5.3.1  Description 

This  technique  suggests  an  approach  to  risk 
planning;  the  process  of  developing  and 
documenting  an  organized,  comprehen¬ 
sive  approach.  It  also  suggests  interactive 
strategy  and  methods  for  identifying  and 
tracking  risk  drivers,  developing  risk-han¬ 
dling  plans,  performing  continuous  assess¬ 
ments  to  determine  how  risks  have 
changed,  and  planning  adequate  resources. 
The  risk  planning  technique  is  applicable 
to  all  functional  areas  in  the  program,  es¬ 
pecially  critical  areas  and  processes.  Using 
the  acquisition  strategy  as  a  starting  point 
results  in  the  development  of  a  program 
risk  management  strategy,  from  which 
flows  a  management  plan  that  provides  the 
detailed  information  and  direction 
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necessary  to  conduct  an  effective  manage¬ 
ment  program.  This  risk  management  plan 
provides  the  PM  with  an  effective  method 
to  define  a  program,  one  that  fixes  respon¬ 
sibility  for  the  implementation  of  its  vari¬ 
ous  aspects,  and  supports  the  acquisition 
strategy. 

The  technique  should  first  be  used  in  Phase 
0  following  the  development  of  the  initial 
acquisition  strategy.  Subsequently,  it  may 
be  used  to  update  the  management  plan 
on  the  following  occasions:  (1)  whenever 
the  acquisition  strategy  changes,  or  there 
is  a  major  change  in  program  emphasis;  (2) 
in  preparation  for  major  decision  points; 
(3)  in  preparation  for  and  immediately  fol¬ 
lowing  technical  audits  and  reviews;  (4) 
concurrent  with  the  review  and  update  of 
other  program  plans;  and  (5)  in  prepara¬ 
tion  for  a  PMO  submission. 


The  PMO  risk  management  coordinator, 
if  assigned,  develops  the  risk  management 
plan  based  on  guidance  provided  by  the 
PM,  and  coordinating  with  the  Program 
Level  IPT.  To  be  effective,  the  PM  must 
make  risk  management  an  important  pro¬ 
gram  management  function  and  must  be 
actively  involved  in  the  risk  planning  ef¬ 
fort.  Planning  requires  the  active  partici¬ 
pation  of  essentially  the  entire  PMO  and 
contractor  team. 

5.3.2  Procedures 

Figure  5-1  graphically  depicts  the  process 
to  be  followed  in  applying  this  technique. 
The  procedure  consists  of  a  number  of  it¬ 
erative  activities  that  result  in  the  devel¬ 
opment  of  the  risk  management  strategy 
and  a  Risk  Management  Plan. 


PM  Guidance 


INPUT 

Acquisition  strategy 

Prior  risk  management 
plan  (if  any) 

Known  risks 
System  description 
Program  description 

Key  ground  rules  and 
assumptions 


Evaluate  risk  planning 
requirements 

Evaluate  the  program’s  current 
risk  situation 

Develop  a  risk  management 
strategy 

Determine  the  tasks  and 
guidance  required  to  implement 
the  risk  management  strategy 

Develop  the  PMO’s  approach  to 
risk  management  in  general 

Provide  application  guidance  for 
risk  management  component 
processes 

Develop  inputs  for  other 
acquisition  strategies  and 
program  processes  _ 


Program-Level  IPT  (or  equivalent  such 
as  Risk  Management  Board) 

Risk  management  coordinator 


OUTPUT 

•  Risk  Management  Plan 

•  Risk  Management 
Training 


Figure  5-1.  Risk  Planning  Technique  Input  and  Output 
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The  acquisition  strategy  and  related  man¬ 
agement  planning  efforts  (program  man¬ 
agement,  and  systems  engineering),  pro¬ 
gram  constraints,  and  any  existing  risk 
management  planning  are  integrated  and 
evaluated  in  the  context  of  the  PM's  guid¬ 
ance,  which  provides  the  direction  for  the 
planning  process.  Typical  types  of  PM 
guidance  are  concerns  about  certain  catego¬ 
ries  of  risk,  guidance  on  funding  of  han¬ 
dling  activities,  emphasis  to  be  placed  on 
risk  management  training,  and  frequency 
and  type  of  internal  reports. 

The  integration  and  evaluation  of  the  pri¬ 
mary  inputs  establish  the  requirements  and 
scope  of  the  planning  effort  through  an  as¬ 
sessment  of  the  program's  current  risk  situ¬ 
ation.  The  results  of  the  assessment  provide 
the  basis  for  development  of  management 
strategy.  The  strategy  should  reflect  the 
level  of  risk  that  the  PM  is  prepared  to  ac¬ 
cept,  and  should  provide  guidance  on  how 
and  when  known  risks  will  be  reduced  to 
acceptable  levels.  It  should  also  describe 
the  risk  management  process  the  PMO  will 
employ  and  the  organization  and  structure 
of  the  management  program,  addressing 
things  such  as  risk  ratings,  the  use  of  an 
MIS,  policy  and  procedures  on  sharing  risk 
management  information,  and  training. 

The  PMO  should  create  an  MIS  early  in  the 
planning  process.  It  will  serve  as  a  plan¬ 
ning  source  and  the  data  may  be  used  for 
creating  reports.  It  will  also  become  the  re¬ 
pository  for  all  current  and  historical  in¬ 
formation  related  to  risk.  Eventually,  this 
information  may  include  risk  assessment 
documents,  contract  deliverables,  if  appro¬ 
priate,  and  other  risk-related  reports. 

Based  on  the  management  strategy,  the 
plan  identifies  specific  tasks  to  be  accom¬ 
plished  and  assigns  responsibility  for  their 
execution.  The  timing  of  these  tasks  should 
be  incorporated  into  an  integrated  critical 


path  master  schedule  or  equivalent.  Guid¬ 
ance  for  task  execution  and  control  should 
also  be  developed,  covering  such  things  as 
the  suggested  techniques  to  be  used  for  each 
component,  any  assistance  available  to  Sub- 
Tier  IPTs,  the  use  of  funds,  the  policy  on  the 
use  of  independent  risk  assessors,  etc.  This 
information  may  be  documented  in  a  risk 
management  plan.  A  sample  format  is 
shown  in  Figure  5-2.  Appendix  B  contains 
two  examples  of  a  Risk  Management  Plan. 

The  contents  of  the  risk  management  strat¬ 
egy  and  plan  should  be  consistent  with 
the  acquisition  strategy  and  other  pro¬ 
gram  plans  derived  from  the  acquisition 
strategy.  Hence,  it  should  be  tailored  to 
each  program  rather  than  attempting  to 
use  the  same  process  and  its  implementa¬ 
tion  on  all  programs.  This  will  help  to  en¬ 
sure  that  risk  is  considered  in  all  program 
activities  and  that  it  does  not  become  a 
"stove  pipe"  function. 

5.4  RISK  ASSESSMENT 
TECHNIQUES 

5.4.1  Product  (WBS)  Risk  Assessment 

5.4. 1.1  Description.  This  technique  iden¬ 
tifies  those  risks  associated  with  a  given 
system  concept  and  design.  The  difference 
between  the  process  (DoD  4245.7-M)  tech¬ 
nique  and  this  approach  is  that  DoD  4245.7- 
M  addresses  the  contractor's  engineering 
and  manufacturing  process  and  this  tech¬ 
nique  focuses  on  the  resulting  product.  This 
technique  is  used  to  identify  and  analyze 
risks  in  the  following  critical  risk  areas: 
design  and  engineering,  technology,  logis¬ 
tics,  production,  concurrency,  plus  others 
as  needed  for  both  hardware  and  software. 

The  WBS  is  the  starting  point  to  describe 
contract  work  to  be  done  and  the  resulting 
product  and  is  the  basis  for  determining 
risk  events  in  each  critical  risk  area.  The 
risk  events — events  that  might  have  a 
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INTRODUCTION.  This  section  should  address  the  purpose  and  objective  of  the  plan,  and  provide  a  brief  summary 
of  the  program,  to  include  the  approach  being  used  to  manage  the  program,  and  the  acquisition  strategy. 

PROGRAM  SUMMARY.  This  section  contains  a  brief  description  of  the  program,  including  the  acquisition  strategy 
and  the  program  management  approach.  The  acquisition  strategy  should  address  its  linkage  to  the  risk 
management  strategy. 

DEFINITIONS.  Definitions  used  by  the  program  office  should  be  consistent  with  DoD  definitions  for  ease  of 
understanding  and  consistency.  However,  the  DoD  definitions  allow  program  managers  flexibility  in  constructing 
their  risk  management  programs.  Therefore,  each  program’s  risk  management  plan  may  include  definitions  that 
expand  the  DoD  definitions  to  fit  its  particular  needs.  For  example,  each  plan  should  include,  among  other 
things,  definitions  for  the  ratings  used  for  technical,  schedule,  and  cost  risk.  (Discussion  of  risk  rating  is  contained 
in  Acquisition  Deskbook,  Section  2.5.2.I.) 

RISK  MANAGEMENT  STRATEGY  AND  APPROACH.  Provide  an  overview  of  the  risk  management  approach, 
to  include  the  status  of  the  risk  management  effort  to  date,  and  a  description  of  the  program  risk  management 
strategy.  See  Acquisition  Deskbook,  Sections  2. 5.2.1  and  2.5.2. 3. 

ORGANIZATION.  Describe  the  risk  management  organization  of  the  program  office  and  list  the  responsibilities 
of  each  of  the  risk  management  participants.  See  Acquisition  Deskbook,  Section  2.5.2.3. 

RISK  MANAGEMENT  PROCESS  AND  PROCEDURES.  Describe  the  program  risk  management  process  to  be 
employed,  i.e.,  risk  planning,  assessment,  handling,  monitoring  and  documentation,  and  a  basic  explanation  of 
these  components.  See  Acquisition  Deskbook,  Section  2.5.2.I.  Also  provide  application  guidance  for  each  of 
the  risk  management  functions  in  the  process.  If  possible,  the  guidance  should  be  as  general  as  possible  to 
allow  the  program’s  risk  management  organization  (e.g.,  IPTs)  flexibility  in  managing  the  program  risk,  yet 
specific  enough  to  ensure  a  common  and  coordinated  approach  to  risk  management.  It  should  address  how  the 
information  associated  with  each  element  of  the  risk  management  process  will  be  documented  and  made 
available  to  all  participants  in  the  process,  and  how  risks  will  be  tracked,  to  include  the  identification  of  specific 
metrics  if  possible. 

RISK  PLANNING.  This  section  describes  the  risk  planning  process  and  provides  guidance  on  how  it  will  be 
accomplished,  and  the  relationship  between  continuous  risk  planning  and  this  RMP.  Guidance  on  updates  of  the 
RMP  and  the  approval  process  to  be  followed  should  also  be  included.  See  Section  2.5.2. 1  of  the  Deskbook  for 
information  on  risk  planning. 

RISK  ASSESSMENT.  This  section  of  the  plan  describes  the  assessment  (identification  and  analysis)  process.  It 
includes  procedures  for  examining  the  critical  risk  areas  and  processes  to  identify  and  document  the  associated 
risks.  It  also  summarizes  the  analyses  process  for  each  of  the  risk  areas  leading  to  the  determination  of  a  risk 
rating.  This  rating  is  a  reflection  of  the  potential  impact  of  the  risk  in  terms  of  its  variance  from  known  Best 
Practices  or  probability  of  occurrence,  its  consequence,  and  its  relationship  to  other  risk  areas  or  processes. 
This  section  may  include: 

•  Overview  and  scope  of  the  assessment  process 

•  Sources  of  information 

•  Information  to  be  reported  and  formats 

•  Description  of  how  risk  information  is  retained 

•  Assessment  techniques  and  tools  (see  Section  2. 5.2.4. 2  of  the  Deskbook). 

RISK  HANDLING.  This  section  describes  the  risk  handling  options,  and  identifies  tools  that  can  assist  in 
implementing  the  risk  handling  process.  It  also  provides  guidance  on  the  use  of  the  various  handling  options  for 
specific  risks. 

RISK  MONITORING.  This  section  describes  the  process  and  procedures  that  will  be  followed  to  monitor  the 
status  of  the  various  risk  events  identified.  It  should  provide  criteria  for  the  selection  of  risks  to  be  reported  on, 
and  the  frequency  of  reporting.  Guidance  on  the  selection  of  metrics  should  also  be  included. 

RISK  MANAGEMENT  INFORMATION  SYSTEM,  DOCUMENTATION  AND  REPORTS.  This  section  describes 
the  MIS  structure,  rules,  and  procedures  that  will  be  used  to  document  the  results  of  the  risk  management 
process.  It  also  identifies  the  risk  management  documentation  and  reports  that  will  be  prepared;  specifies  the 
format  and  frequency  of  the  reports;  and  assigns  responsibility  for  their  preparation. 


Figure  5-2.  Sample  Format  for  Risk  Management  Plan 
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detrimental  impact  on  the  system,  sub¬ 
systems,  or  components — are  evaluated  to 
identify  and  characterize  specific  risks  rat¬ 
ings  and  prioritization. 

This  technique  should  be  used  shortly  af¬ 
ter  the  completion  of  the  prime  contractor's 
WBS.  Thereafter,  it  should  be  used  regu¬ 
larly  up  to  the  start  of  production.  The  tech¬ 
nique  can  be  used  independently  or  in  con¬ 
junction  with  other  risk  assessment  tech¬ 
niques,  such  as  the  Process  (DoD  4245.7- 
M)  Risk  Assessment  technique.  It  may,  if 
appropriate,  also  be  used  in  conjunction 
with  the  Integrated  Baseline  Review  (IBR), 
which  is  conducted  within  6  months  of  con¬ 
tract  award.  See  Section  1.4.2.4.3  of  the 
Deskbook  (http://www.deskbook.osd.mil) 
for  a  discussion  of  IBR.  A  World  Wide  Web 
Site  is  also  available  at  www.acq.osd.rniL/ 
pm/ibrmats.htm,  which  discusses  the  IBR 
Process. 

To  apply  this  technique,  joint  Government 
and  industry  evaluation  teams  should  ex¬ 
amine  the  appropriate  WBS  levels  in  each 
Sub-Tier  IPTs  product  area.  If  necessary, 
complementary  industry-only  teams  may 
take  an  in-depth  look  at  selected  areas  at 


lower  WBS  levels.  At  times,  it  may  be  de¬ 
sirable  to  include  outside  industry  experts 
on  the  teams  to  aid  in  the  examination  of 
specific  WBS  elements  or  functional  areas. 

5.4.1.2  Procedures.  Figure  5-3  depicts  the 
process  used  in  this  technique.  The  first 
step  is  to  review  the  WBS  elements  down 
to  the  level  being  considered,  and  identify 
risk  events.  This  review  should  consider 
the  critical  areas  (design  and  engineering, 
technology,  logistics,  etc.)  that  may  help  to 
describe  risk  events.  Table  5-1  shows  a  par¬ 
tial  listing  of  these  elements. 

Using  information  from  a  variety  of 
sources,  such  as  program  plans,  prior  risk 
assessments,  expert  interviews,  etc.,  the 
WBS  elements  are  examined  to  identify 
specific  risks  in  each  critical  area.  The  risk 
event,  are  then  analyzed  to  determine 
probability  of  occurrence  and  conse¬ 
quences,  along  with  any  interdependencies 
and  risk  event  priorities.  Several  tech¬ 
niques  and  tools  are  available  to  accom¬ 
plish  this,  including,  among  others,  tech¬ 
nology  assessments,  modeling  and 
simulation,  hazard  analysis,  and  fault  tree 
analysis. 


INPUT 

•  WBS 

•  Integrated  Master  Schedule  (or 
equivalent) 

•  Critical  Area  Evaluation  Criteria 

OUTPUT 

•  Program  Plans 

•  Examine  WBS  elements  and 

•  Risk  Information  Forms 

•  Past  Projected  Data 

identify  risk  events 

•  Lesson  Learned  - >• 

•  Analyze  risk  events 

- >■  •  Prioritized  List  of  Risks 

•  Expert  Interview  Data 

(Includes  rated  and 

•  Critical  Area  Risk 

•  Test  Results 

prioritized  risk  events) 

Evaluations 

•  Integrated  Baseline 

Review 

t 

•  Sub-Tier  IPT  Evaluation  Teams 

•  “Outside”  Industrial  Experts 

Figure  5-3.  Product  (WBS)  Risk  Assessment  Technique  Input  and  Output 


53 


Critical  Risk 
Areas 

Example  Elements 

Design  and 
Engineering 

Design/technology  approach  Integration  requirements 

Operational  environments  Human-machine  interface 

External/internal  interfaces  Design  growth  capacity 

Use  of  standard  parts/program  parts  list  Design  maturity 

System/subsystem  critical  design  Safety  &  health  hazards 

requirement  Manpower,  training  and  skill  profiles 

Logistics 

Operations  and  Maintenance  (O&M)  Support  equipment  requirements 

concept  Maintenance  interfaces 

System  diagnostic  requirement  l_evei  0f  repair  decisions 

Repairability  and  Maintainability  (R&M)  Training  equipment  design 
requirements 

Supply  support  requirements 

Built-in  Test  (BIT)  requirements 

Testing 

Integrated  test  Test  environmental  acceleration 

Qualification  testing  Supportability  test  results 

Subsystem  test  limits 

Manufacturing 

Design  producibility  Special  tooling/test  equipment  planning 

personnel  availability 

Manufacturing  capability  requirements  Process/tooling  proofing 

Parts/assemblies  availability  Production  equipment  availability 

Concurrency 

Program  schedule  adequacy  Development  phases  concurrency 

Table  5-1.  Critical  Risk  Areas  and  Example  Elements 

The  results  of  this  analysis  should  be  docu¬ 
mented  in  a  program-specific  standard  for¬ 
mat,  such  as  a  Risk  Information  Form  (RIF). 
The  risks,  along  with  others  identified  us¬ 
ing  other  techniques,  can  be  prioritized  and 
aggregated  using  the  technique  described 
later  in  this  chapter. 

5.4.2  Process  (DoD  4245.7-M) 

Risk  Assessment 

5.4.2.1  Description.  This  technique  is  used 
to  assess  (identify  and  analyze)  program 
technical  risks  resulting  from  the  con¬ 
tractor's  processes.  It  is  based  on  the 
application  of  the  technical  risk  area 
templates  found  in  DoD  4245.7-M.  These 


templates  describe  the  risk  areas  contained 
in  the  various  technical  processes  (e.g.,  de¬ 
sign,  test,  production,  etc.)  and  specify 
methods  for  reducing  risks  in  each  area. 
Success  of  any  risk  reduction  efforts  asso¬ 
ciated  with  this  technique  will  depend  on 
the  contractor's  ability  and  willingness  to 
make  a  concerted  effort  to  replace  any  de¬ 
ficient  engineering  practices  and  proce¬ 
dures  with  best  industrial  practices. 

One  of  the  primary  benefits  of  this  tech¬ 
nique  is  that  it  addresses  pervasive  and 
important  sources  of  risk  in  most  DoD  ac¬ 
quisition  programs  and  uses  fundamental 
engineering  principles  and  proven  proce¬ 
dures  to  reduce  technical  risks.  The 
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technique  is  accepted  by  many  aerospace 
companies  in  normal  business  activities, 
and  in  fact,  was  developed  by  a  group  of 
Government  and  aerospace  experts. 

The  technique  is  primarily  applicable  dur¬ 
ing  Phases  0,  and  II  of  program  develop¬ 
ment.  In  Phase  0  it  provides  a  detailed 
checklist  of  processes  that  the  contractor 
needs  to  address;  in  Phase  II,  the  processes 
are  being  implemented  in  Low  Rate  Initial 
Production  (LRIP).  The  description  of  each 
template  in  DoD  4245. 7-M  shows  the 
phases  in  which  the  template  should  be  ap¬ 
plied.  The  specific  timing  of  the  applica¬ 
tion  within  the  phases  should  be  deter¬ 
mined  based  on  the  type  of  program,  the 
acquisition  strategy  and  plans,  and  the 
judgment  of  program  officials.  It  should 
also  be  used  in  preparation  for  milestone 
decisions  and  when  preparing  for  source 
selection.  This  technique  may  be  used 


independently  or  in  conjunction  with  other 
risk  assessment  techniques.  When  feasible, 
a  Government-industry  evaluation  team 
should  be  formed  early  in  the  program  to 
apply  this  technique. 

5.4.2.2  Procedures.  Figure  5-4  shows  the 
basic  approach  used  in  this  technique.  The 
DoD  4245.7-M  templates  are  used  in  con¬ 
junction  with  the  contract  requirements 
and  specifications  to  identify  those  techni¬ 
cal  processes  critical  to  the  program  and  to 
establish  a  program  baseline  of  contractor 
processes.  When  possible,  the  program 
baseline  should  be  determined  by  evalu¬ 
ating  actual  contractor  performance,  as 
opposed  to  stated  policy.  For  example,  de¬ 
sign  policy  should  be  determined  from  in¬ 
terviewing  designers  and  not  simply  from 
reviewing  written  corporate  policies. 

This  program  baseline  should  then  be 
compared  to  a  baseline  of  industry-wide 


•  Corporate  Policies,  Practices 
&  Procedures 

•  Contract  Requirements 
Specifications  & 

Modifications 

I 

INPUT 

1 

•  DoD  4245.7-M 

•  Identify  Program’s  Critical 
Technical  Processes 

OUTPUT 

Templates 

•  Combined  Government/ 

Industry  Acquisition  Flow _ y, 

Chart 

•  Develop  Technical  Baseline 
for  Critical  Technical 

Processes 

•  Develop  Program  Baseline 

•  Technical  Baseline 
y  •  Proaram  Baseline 

•  Risk  Information  Forms 

•  Known  Best  Practices 

•  Past  Project  Data 

•  Measure  Variances  Between 
Baselines 

•  Technical  Risk 
Assessment  Summary 

•  Best  Practices  Database 

•  Report  Risks 

(PMWS) 

t 

•  Government-Industrial 
Evaluation  Team 

•  “Outside”  Industrial 

Experts 

Figure  5-4.  Process  (DoD  4245.7-M)  Risk  Assessment  Technique  Input  and  Output 
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processes  and  practices  that  are  critical  to 
the  program.  The  baseline  should  be  de¬ 
veloped  by  reviewing  and  compiling 
known  best  practices  in  use  by  various 
companies  in  both  defense  and  non-de¬ 
fense  sectors.  One  source  of  best  practices 
information  is  the  Program  Manager's 
Work  Station  (PMWS),  a  series  of  PC  ex¬ 
pert  systems  designed  to  aid  in  the  imple¬ 
mentation  of  DoD  4245.7-M.  The  point  of 
contact  for  the  PMWS  is  the  Best  Man¬ 
agement  Practices  Center  of  Excellence 
(http://www.bmpcoe.org). 

The  differences  between  the  two  baselines 
are  a  reflection  of  the  technical  process  risk 
present.  These  results  should  be  docu¬ 
mented  in  a  standard  format,  such  as  a  pro¬ 
gram-specific  Risk  Information  Form  (see 
MIS  discussion  this  section)  to  facilitate  the 
development  of  a  risk  handling  and  risk 
reporting  plan. 

5.4.3  Program  Documentation 

Evaluation  Risk  Identification 

5.4.3.1  Description.  This  technique  pro¬ 
vides  a  methodology  for  comparing  key 
program  documents  and  plans  to  ensure 
that  they  are  consistent  and  traceable  to  one 
another.  Program  documents  and  plans  are 
hierarchical  in  nature.  If  the  contents  (ac¬ 
tivities,  events,  schedules,  requirements, 


specifications,  etc.)  of  a  document  or  plan 
do  not  flow  from  or  support  the  contents 
of  those  above,  below,  or  adjacent  to  it, 
there  is  a  strong  chance  that  risk  will  be 
introduced  into  the  program  or  that 
known  risks  will  not  be  adequately  ad¬ 
dressed.  This  technique  reduces  those 
risks  and  improves  the  quality  of  program 
documentation. 

This  technique  can  be  used  in  any  acquisi¬ 
tion  phase  as  documents  or  plans  are  be¬ 
ing  developed  or  updated.  The  compari¬ 
son  of  program  documentation  and  plans 
should  be  performed  by  a  small  team  of 
experienced,  knowledgeable  personnel 
who  are  intimately  familiar  with  the  total 
program. 

5.4.3.2  Procedures.  Figure  5-5  shows  the 
process  used  in  this  technique.  The  primary 
inputs  to  the  process  are  the  PMO  docu¬ 
ments  that  detail  the  steps  involved  in  ex¬ 
ecuting  the  program.  These  include,  for 
example,  the  Mission  Need  Statement 
(MNS),  ORD,  acquisition  plan,  any  master 
management  plan.  Test  and  Evaluation 
Master  Plan  (TEMP),  manufacturing  plan, 
etc.  Another  set  of  key  input  documents 
are  those  used  to  communicate  with  the 
prime  contractor,  e.g.,  WBS,  specifications. 
Statement  of  Work  (SOW)  or  equivalent 
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Are  high  risk  performance 
specifications  being  tested 
in  a  manner  to  support  risk 
reduction? 

Figure  5-6.  Phase  0  Correlation  of  Selected  Documents  (Example) 

such  as.  Statement  of  Objectives,  etc.  Be-  identify  areas  that  are  not  critical  and  are 

fore  any  comparison,  the  PMO  should  re-  available  for  trade  to  meet  cost  objectives, 

view  all  documents  for  accuracy  and  com-  Risk  is  a  factor  in  CAIV  considerations, 
pleteness.  Figure  5-6  shows  an  example  of 
the  type  of  correlation  that  should  exist 
among  the  MNS,  ORD,  and  TEMP  during 
Phase  0. 

If  the  comparison  shows  any  gaps  or 
inconsistencies,  reviewers  should  identify 
them  as  possible  risks  on  a  Risk  Identifica¬ 
tion  Form  (RIF),  the  output  of  this  process. 

5.4.4  Threat  and  Requirements  Risk 
Assessment 

5.4.4.1  Description.  This  technique  de¬ 
scribes  an  approach  to  assess  risks  associ¬ 
ated  with  requirements  and  threat  and  to 
identify  requirements  and  threat  elements 
that  are  risk  drivers.  Because  operational 
needs,  environmental  demands,  and 
threat  determine  system  performance  re¬ 
quirements,  to  a  large  degree,  they  are  a 
major  factor  in  driving  the  design  of  the 
system  and  can  introduce  risk  in  a  pro¬ 
gram.  Further,  with  the  introduction  of 
CAIV,  PMs  and  users  are  directed  to 
examine  performance  requirements  and 


The  requirements  risk  assessment  process 
focuses  on:  determining  if  operational  re¬ 
quirements  are  properly  established  and 
clearly  stated  for  each  program  phase;  en¬ 
suring  that  requirements  are  stable  and  the 
operating  environment  is  adequately 
described;  addressing  logistics  and  suit¬ 
ability  needs;  and  determining  if  require¬ 
ments  are  too  constrictive,  thereby  identi¬ 
fying  a  specific  solution.  The  evaluation 
of  the  threat  risk  assessment  process'  ma¬ 
turity  addresses:  uncertainty  in  threat  ac¬ 
curacy  and  stability,  sensitivity  of  design 
and  technology  to  threat,  vulnerability  of 
the  system  to  threat  countermeasures,  and 
vulnerability  of  the  program  to  intelli¬ 
gence  penetration.  PMs  should  view  re¬ 
quirements  in  the  context  of  the  threat  and 
accurately  reflect  operational,  environ¬ 
mental,  and  suitability  requirements  in 
design  documents. 

PMs  should  use  threat  and  requirements 
assessments  during  the  early  phases  of 
program  development  and,  as  necessary. 
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Figure  5-7.  Threat  and  Requirement  Risk  Assessment  Technique  Input  and  Output 


as  the  program  advances  through  de¬ 
velopment.  Early  and  complete  under¬ 
standing  of  the  requirements  and  threat 
precludes  misunderstandings  between  the 
requirements  and  development  communi¬ 
ties,  helps  to  identify  risk  areas,  and  allows 
early  planning  to  handle  risk.  Conse¬ 
quently,  the  user  should  be  actively  in¬ 
volved  in  this  process  from  the  beginning. 

5.4.4.2  Procedures.  Figure  5-7  depicts  the 
process  used  in  this  technique.  The  basic 
approach  is  to  conduct  a  thorough  review 
of  the  documents  containing  performance 
requirements  and  threat  information,  e.g., 
ORD,  TEMP,  System  Specification,  Special 
Threat  Assessment  (STA),  Design  Refer¬ 
ence  Mission  Profile,  etc.,  to  determine  sta¬ 
bility,  accuracy,  operating  environment, 
logistics  and  suitability  requirements,  and 
consistency  between  these  requirements 
and  the  threat  considerations  cited  above. 
There  should  be  an  understanding  be¬ 
tween  the  users  and  the  developers  on  Key 
Performance  Parameters  (KPPs)  in  order 
to  identify  the  requirements  that  are  most 
important  and  critical  to  program  success. 


The  Design  Reference  Mission  Profile  and 
Design  Requirements  templates  in  DoD 
4245.7-M  and  the  Program  Documenta¬ 
tion  Evaluation  Risk  Identification  tech¬ 
nique  may  be  useful  in  support  of  this 
technique. 

Requirements  should  be  thoroughly  re¬ 
viewed  to  identify  those  that  drive  perfor¬ 
mance.  This  will  require  the  "flow  down" 
of  performance  requirements  to  compo¬ 
nents  and  subassemblies  and  the  identifi¬ 
cation  of  technologies /techniques  to  be 
used  in  these  components /subassemblies 
that  may  significantly  affect  the  system's 
ability  to  meet  users'  needs. 

Designers  should  determine  the  sensitiv¬ 
ity  of  system  performance  to  the  require¬ 
ments  and  threat  and  identify  risk  drivers. 
Models  and  simulations  are  useful  tools  to 
determine  this  sensitivity.  For  example,  the 
U.S.  Army  Materiel  System  Analysis  Ac¬ 
tivity  (AMSAA)  has  such  an  analytic 
model,  the  AMSAA  Risk  Assessment  Meth¬ 
odology.  The  AMSAA  point  of  contact  can 
be  reached  at  (410)  278-6626. 
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The  PMWS  can  also  be  useful.  The  risk 
identified  in  this  technique  should  be 
documented  in  a  program-specific  format, 
such  as  a  Risk  Information  Form  (RIF)  (see 
Annex  B). 

5.4.5  Cost  Risk  Assessment 

5.4.5.1  Description.  This  technique  pro¬ 
vides  a  program-level  cost  estimate  at 
completion  (EAC)  that  is  a  function  of  per¬ 
formance  (technical),  and  schedule  risks. 
It  uses  the  results  of  previous  assessments 
of  WBS  elements  and  cost  probability  dis¬ 
tributions  developed  for  each  of  the  ele¬ 
ments.  These  individual  WBS  elements  are 
aggregated  using  a  Monte  Carlo  simula¬ 
tion  to  obtain  a  probability  distribution  of 
the  program-level  cost  EAC  probability 
distribution  function.  These  results  are 
then  analyzed  to  determine  the  actual  risk 
of  cost  overruns  and  to  identify  the  cost 
drivers. 

The  use  of  these  cost  probability  distribu¬ 
tions  as  the  basis  for  the  program-level  cost 
estimate  results  in  a  more  realistic  EAC 
than  the  commonly  used  single  point  esti¬ 
mates  for  WBS  elements,  since  they  address 


both  the  probability  of  occurrence  and 
consequences  of  potential  risk  events.  Their 
use  also  eliminates  a  major  cause  of  under¬ 
estimating  (use  of  point  estimates)  and  per¬ 
mits  the  evaluation  of  performance  (tech¬ 
nical)  or  schedule  causes  of  cost  risk.  Thus, 
this  technique  provides  a  basis  for  the  de¬ 
termination  of  an  "acceptable"  level  of  cost 
risk. 

This  technique  can  be  used  in  any  of  the 
acquisition  phases,  preferably  at  least  once 
per  phase  beginning  in  Phase  0  although 
suitable  data  may  not  exist  until  Phase  I  in 
some  cases.  It  should  be  used  in  conjunc¬ 
tion  with  performance  (technical)  and 
schedule  risk  assessments  and  may  be 
performed  by  small  Government-industry 
teams  consisting  of  risk  analysts,  cost  ana¬ 
lysts,  schedule  analysts  and  technical  ex¬ 
perts  who  understand  the  significance  of 
previous  performance  and  schedule  risk 
assessments.  They  should  report  to  the  Pro¬ 
gram  IPT.  This  technique  requires  close  and 
continuous  cooperation  among  cost  ana¬ 
lysts  and  knowledgeable  technical  person¬ 
nel  and  the  support  of  the  prime 
contractor's  senior  management  to  help  get 
valid  cost  data. 
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5A.5.2  Procedures.  Figure  5-8  depicts  the 
process  used  in  applying  this  technique. 
The  first  step  is  to  identify  the  lowest 
WBS  level  for  which  cost  probability  dis¬ 
tribution  will  be  constructed.  The  level 
selected  will  depend  on  the  program 
phase;  e.g.,  during  Phase  0,  it  may  not  be 
possible  to  go  beyond  level  2  or  3,  simply 
because  the  WBS  has  not  yet  been  devel¬ 
oped  to  lower  levels.  As  the  program  ad¬ 
vances  into  subsequent  phases  and  the 
WBS  is  expanded,  it  will  be  possible  and 
necessary  to  go  to  lower  levels  (4,  5,  or 
lower).  Specific  performance  (technical) 
and  schedule  risks  are  then  identified  for 
these  WBS  elements. 

To  develop  the  WBS  elements  cost  prob¬ 
ability  distributions,  the  team,  working 
with  the  prime  contractor's  WBS  element 
managers,  determines  the  cost  range  for 
each  element  being  investigated.  The  cost 
range  encompasses  cost  estimating  uncer¬ 
tainty,  schedule  risk,  and  technical  risk. 
The  validity  of  the  cost  data  used  to  con¬ 
struct  the  distribution  is  critical.  In  fact, 
collecting  good  data  is  the  largest  part  of 
the  cost  risk  job.  Consequently,  PMOs 
should  place  major  emphasis  on  this 
effort. 

The  element  cost  probability  distributions 
aggregated  and  evaluated  are  then  using  a 
Monte  Carlo  simulation  program.  All 
Monte  Carlo  processes  contain  limitations, 
but  they  are  more  informative  than  point 
estimates.  Any  number  of  these  simula¬ 
tions  are  readily  available  to  perform  this 
aggregation,  and  one  that  meets  the  spe¬ 
cific  needs  of  the  program  should  be  se¬ 
lected.  The  results  of  this  step  will  be  a  pro¬ 
gram-level  cost  EAC  and  a  cost  distribu¬ 
tion  that  shows  the  cumulative  probabil¬ 
ity  associated  with  different  cost  values. 
These  outputs  are  then  analyzed  to  deter¬ 
mine  the  level  of  cost  risk  and  to  identify 


the  specific  cost  drivers.  Cost  risk  is  deter¬ 
mined  by  comparing  the  EAC  with  the  cost 
baseline  developed  as  part  of  the  acquisi¬ 
tion  program  baseline.  Since  the  EAC  and 
program  cost  distribution  are  developed 
from  WBS  element  risk  assessments,  it  is 
possible  to  determine  the  cost  risk  drivers. 
The  cost  drivers  can  also  be  related  back  to 
the  appropriate  performance  and  schedule 
risks.  The  results  of  the  analysis  (cost  risks 
and  drivers)  should  be  documented  in 
RIFs. 

5.4.6  Quantified  Schedule  Risk 
Assessment 

5.4.6.1  Description.  This  technique  pro¬ 
vides  a  means  to  determine  program-level 
schedule  risk  as  a  function  of  risk 
associated  with  various  activities  that  com¬ 
pose  the  program.  It  estimates  the  pro¬ 
gram-level  schedule  by  developing  prob¬ 
ability  distributions  for  each  activity  du¬ 
ration  and  aggregating  these  distributions 
using  a  Monte  Carlo  simulation  or  other 
analytical  tools.  The  resulting  program- 
level  schedule  is  then  analyzed  to  deter¬ 
mine  the  actual  schedule  risk  and  to  iden¬ 
tify  the  schedule  drivers. 

This  technique  expands  the  commonly 
used  Critical  Path  Method  (CPM)  of  de¬ 
veloping  a  program  schedule  to  obtain  a 
realistic  estimate  of  schedule  risk.  The  ba¬ 
sic  CPM  approach  uses  single  point  esti¬ 
mates  for  the  duration  of  program  activi¬ 
ties  to  develop  the  program's  expected 
duration  and  schedule.  It  invariably  leads 
to  underestimating  the  time  required  to 
complete  the  program  and  schedule  over¬ 
runs,  primarily  because  the  point  esti¬ 
mates  do  not  adequately  address  the  un¬ 
certainty  inherent  in  individual  activities. 
The  uncertainty  can  be  caused  by  a  num¬ 
ber  of  factors  and  may  be  a  reflection  of 
the  risk  present  in  the  activity. 
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The  quantified  schedule  technique  ac¬ 
counts  for  uncertainty  by  using  a  range 
of  time  that  it  will  take  to  complete  each 
activity  instead  of  single  point  estimates. 
These  ranges  are  then  combined  to  deter¬ 
mine  the  program-level  schedule  esti¬ 
mate.  This  approach  enables  PMs  to  esti¬ 
mate  early  in  a  program  if  there  is  a  sig¬ 
nificant  likelihood  of  overrunning  the  pro¬ 
gram  schedule  and  by  how  much.  It  also 
identifies  program  activities  that  are  on 
the  "highest  risk  path." 

This  technique  can  be  used  in  any  acqui¬ 
sition  phase  beginning  with  the  comple¬ 
tion  of  the  first  statement  of  work.  The 
schedule  probability  distribution  func¬ 
tion  for  each  key  activity  should  be  de¬ 
veloped  as  soon  as  the  activity  is  in¬ 
cluded  in  the  master  schedule.  The  dis¬ 
tribution  functions  should  be  periodi¬ 
cally  reviewed  and  revised,  if  necessary, 
at  least  once  per  phase.  The  technique 
should  be  applied  by  a  small  Govern¬ 
ment-industry  team  consisting  of  sched¬ 
ule  analysts  and  technical  experts  who 
understand  the  significance  of  prior  risk 
performance  assessments. 


5.4.6.2  Procedures.  Figure  5-9  shows  the 
process  used  in  this  technique.  The  first 
step  is  to  identify  the  lowest  activity  level 
for  which  duration/ schedule  probability 
distribution  functions  will  be  constructed. 
The  WBS  should  be  used  as  the  starting 
point  for  identifying  activities  and  con¬ 
structing  a  network  of  activities.  The  WBS 
level  selected  will  depend  on  the  program 
phase. 

Next,  the  contractor  should  construct  a 
CPM  schedule  for  these  activities.  To  de¬ 
velop  the  activity  duration  probability  dis¬ 
tribution  functions,  the  team,  working  with 
the  prime  contractor's  WBS  element  man¬ 
agers,  determines  and  analyzes  duration 
range  for  each  activity  being  investigated. 
This  analysis  should  be  done  by  schedule 
analysts  working  closely  with  knowledge¬ 
able  technical  people. 

The  activity  duration  probability  distribu¬ 
tions  are  aggregated  using  a  Monte  Carlo 
simulation  program,  such  as  ©Risk,  Risk  + 
for  Microsoft  Project,  or  Crystal  Ball.  The 
result  of  this  step  is  a  program-level  sched¬ 
ule  and  distribution  function  that  shows 
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Figure  5-9.  Schedule  Risk  Assessment  Technique  Input  and  Output 
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the  cumulative  probability  associated  with 
different  duration  values.  These  outputs 
are  then  analyzed  to  determine  the  level 
of  schedule  risk  and  to  identify  the  spe¬ 
cific  schedule  drivers.  Risk  is  determined 
by  comparing  the  program-level  schedule 
with  the  deterministic  schedule  baseline 
developed  as  part  of  the  acquisition  pro¬ 
gram  baseline.  The  fact  that  the  schedule 
and  distribution  are  developed  from  WBS 
element  risk  assessments  makes  it  possible 
to  determine  the  schedule  risk  drivers. 
These  drivers  can  also  be  related  back  to 
the  appropriate  performance  risks.  The  re¬ 
sults  of  the  analysis  (schedule  risks  and 
drivers)  should  be  documented  in  RIFs. 
The  analysis  requires  continued  close  co¬ 
operation  between  the  schedule  analysts 
and  technical  personnel  familiar  with  the 
details  of  the  program. 

5.4.7  Expert  Interviews 

5.4.7.1  Description.  A  difficult  part  of  the 
risk  management  process  is  data  gather¬ 
ing.  This  technique  provides  a  means  for 
collecting  risk-related  data  from  subject- 
matter  experts  and  from  people  who  are 
intimately  involved  with  the  various  as¬ 
pects  of  the  program.  It  relies  on  "expert" 


judgment  to  identify  and  analyze  risk 
events,  develop  alternatives,  and  provide 
"analyzed"  data.  It  is  used  almost  exclu¬ 
sively  in  a  support  role  to  help  develop 
technical  data,  such  as  probability  and  con¬ 
sequences  information,  required  by  a  pri¬ 
mary  risk  assessment  technique.  It  can  ad¬ 
dress  all  the  functional  areas  that  make  up 
the  critical  risk  areas  and  processes,  and 
can  be  used  in  support  of  risk  handling. 

Expert  judgment  is  a  sound  and  practical 
way  of  obtaining  necessary  information 
that  is  not  available  elsewhere  or  practical 
to  develop  using  engineering  or  scientific 
techniques.  However,  interviewers  should 
be  aware  that  expert  opinions  may  be  bi¬ 
ased  because  of  over-reliance  on  certain  in¬ 
formation  and  neglect  of  other  information; 
unwarranted  confidence;  the  tendency  to 
recall  most  frequent  and  most  recent 
events;  a  tendency  to  neglect  rare  events; 
and  motivation.  Results  may  have  to  be 
tempered  because  of  these  biases. 

5.4.7.2  Procedures.  Figure  5-10  depicts  the 
process  used  in  this  technique.  The  first 
step  in  the  process  is  to  identify  risk  areas 
and  processes  that  are  to  be  evaluated  us¬ 
ing  the  expert  interview  technique.  Other 
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techniques  described  in  this  section  (e.g., 
WBS  Risk  Assessment,  Process  Risk  Assess¬ 
ment,  etc.)  can  be  used  for  this  purpose. 

Once  the  areas  and  processes  are  known, 
subject-matter  experts  and  program/ 
contractor  personnel  knowledgeable  of  the 
areas  and  processes  should  be  identified 
to  be  interviewed.  Similarly,  qualified  in¬ 
terviewers  should  be  selected  for  each  area 
and  process. 

Interviewers  should  prepare  themselves 
by  preparing  a  strategy  and  selecting  a 
methodology  for  analysis  and  quantifica¬ 
tion  of  data.  The  references  list  sources  for 
practical  techniques  for  quantifying  expert 
judgment. 

After  the  interview,  evaluators  analyze  the 
data  for  consistency,  resolve  any  issues, 
and  document  the  results.  Commercial 
"Groupware"  software  is  available  to  as¬ 
sist  in  compiling  and  documenting  the  re¬ 
sults  of  interviews. 

5.4.8  Analogy  Comparison/ 

Lessons-Leamed  Studies 

5.4.8.1  Description.  This  technique  uses 
lessons  learned  and  historical  information 
about  the  risk  associated  with  programs 
that  are  similar  to  the  new  system  to  iden¬ 
tify  the  risk  associated  with  a  new  program. 
It  is  normally  used  to  support  other  pri¬ 
mary  risk  assessment  techniques,  e.g., 
Product  (WBS)  Risk  Assessment,  Process 
Risk  Assessment,  etc.  The  technique  is 
based  upon  the  concept  that  "new"  pro¬ 
grams  are  originated  or  evolved  from  ex¬ 
isting  programs  or  simply  represent  a  new 
combination  of  existing  components  or 
subsystems.  This  technique  is  most  appro¬ 
priate  when  systems  engineering  and  sys¬ 
tems  integration  issues,  plus  software  de¬ 
velopment,  are  minimal.  A  logical  exten¬ 
sion  of  this  premise  is  that  key  insights  can 


be  gained  concerning  aspects  of  a  current 
program's  risks  by  examining  the  suc¬ 
cesses,  failures,  problems,  and  solutions  of 
similar  existing  or  past  programs.  This 
technique  addresses  all  the  functional  ar¬ 
eas  that  make  up  the  critical  risk  areas  and 
processes. 

5.4.8.2  Procedures.  Figure  5-11  depicts  the 
process  used  in  this  technique.  The  first 
step  in  this  approach  is  to  select  or  develop 
a  baseline  comparison  system  (BCS)  that 
closely  approximates  the  characteristics  of 
the  new  system /equipment  to  as  low  a 
level  as  possible  and  uses  the  processes 
similar  to  those  that  are  needed  to  develop 
the  new  system.  For  processes,  industry¬ 
wide  best  practices  should  be  used  as  a 
baseline.  The  PMWS  is  a  useful  tool  for 
identifying  these  best  practices. 

Relevant  BCS  data  are  then  collected,  ana¬ 
lyzed,  and  compared  with  the  new  system 
requirements.  The  BCS  data  may  require 
adjustment  to  make  a  valid  comparison;  for 
example,  apply  appropriate  inflation  indi¬ 
ces  for  cost  comparisons,  adjust  design 
schedule  for  software  evolution  versus 
software  development,  etc.  The  compari¬ 
sons  can  be  a  major  source  of  risk  assess¬ 
ment  data  and  provide  some  indication  of 
areas  that  should  be  investigated  further. 

5.5  RISK  PRIORITIZATION 

5.5.1  Description 

This  technique  provides  a  means  to  priori¬ 
tize  the  risks  present  in  a  program.  It  is  a 
part  of  risk  analysis.  The  prioritized  list 
provides  the  basis  for  developing  handling 
plans,  preparing  a  handling  task  sequence 
list,  and  allocating  handling  resources. 

When  using  this  technique,  PMs  establish 
definitive  criteria  to  evaluate  the  risks,  such 
as,  probability  (likelihood)  of  failure,  (PF), 
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•  Primary  risk  assessment  technique 
requirements  and  constraints 

I 


OUTPUT 


•  Risk  assessment  data 


t 

•  Government-industrial  Evaluation  Team  _ 

Figure  5-11.  Analogy  Comparison/Lessons  Learned  Studies  Top-Level  Diagram 


INPUT 

Description  and  program 
characteristics  of  the 
new  system  and  its 
components 

Information  needs 


Identify  Baseline  Comparison 
System  (BCS)  candidates 

Breakdown  new  system  into 
logical  components  for 
comparison 

Select  BCS 

Collect  detailed  data  on  BCS 
and  new  system 

Analyze  BCS  and  new  system 
data 

Conduct  comparisons  and 
develop  conclusions 


and  consequence  of  failure  (CF),  along 
with  any  other  factors  considered  appro¬ 
priate.  The  risks  are  evaluated  using  quali¬ 
tative  expert  judgment  and  multi-voting 
methods  to  prioritize  and  aggregate  risks. 
(See  References-SEI,  Continuous  Risk  Man¬ 
agement,  1996,  for  a  discussion  of  multi¬ 
voting  methods.)  A  qualitative  approach 
using  subject-matter  experts  is  generally 
preferred  in  this  technique  because  of  the 
tendency  to  rely  on  ordinal  values  to  de¬ 
scribe  Pc,  CE  and  the  inherent  inaccuracies 
resulting  from  any  attempts  to  use  quanti¬ 
fiable  methods  with  derived  from  raw  (un¬ 
calibrated)  ordinal  scales. 

These  techniques  should  be  used  appropri¬ 
ately  during  Phases  0, 1,  and  II,  at  the  con¬ 
clusion  of  a  major  risk  assessment  under¬ 
taking,  when  there  has  been  a  significant 
change  in  the  acquisition  strategy,  when 
risk  monitoring  indicates  significant 
changes  in  the  status  of  a  number  of  risks, 
and  prior  to  a  milestone  review. 


The  PMO  risk  management  coordinator  (if 
assigned)  may  function  as  a  facilitator  and 
support  the  program  IPT  in  applying  these 
techniques. 

5.5.2  Procedures 

Figure  5-12  depicts  the  process  used  to  pri¬ 
oritize  the  risks  present  in  a  program.  The 
inputs  of  this  process  are  risks  that  have 
been  identified. 

The  evaluation  team,  through  consensus  or 
as  directed  by  the  Risk  Management  Plan, 
selects  the  prioritization  criteria.  PF  and  CF 
should  always  be  part  of  the  criteria,  along 
with  any  other  appropriate  factors.  Ur¬ 
gency,  an  indication  of  the  time  available 
before  the  procedures  for  handling  the  spe¬ 
cific  risk  must  be  initiated,  is  often  consid¬ 
ered  in  the  evaluation.  The  PM  may  also 
choose  to  rank-order  the  prioritization  cri¬ 
teria,  e.g.,  consequence  is  more  important 
than  probability. 
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•  Risk  Prioritization  List 
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•  Program-Level  IPT 

•  Risk  Management  Coordination 

Figure  5-12.  Risk  Prioritization  Technique  Input  and  Output 


A  multi-voting  method  is  useful  to  priori¬ 
tize  risks  (see  References-Scholtes,  1988; 
Linstone,  1975).  The  Delphi  method  is  a 
simple  and  effective  method  of  arriving 
at  a  consensus  among  a  group  of  experts. 
The  procedure  is  for  team  members  to  vote 
on  the  priority  of  each  risk  and  tally  the 
results,  which  are  fed  back  to  the  team. 
Team  members  vote  again  and  the  process 
is  repeated  until  no  changes  occur  in  the 
results.  It  is  normal  to  reach  the  final  out¬ 
come  within  a  few  voting  sessions.  If  there 
are  a  large  number  of  risks,  they  may  be 
broken  into  smaller  groups  for  ranking. 
As  a  general  rule,  no  more  than  10  items 
should  be  prioritized  per  vote.  The  results 


of  the  series  of  votes  are  documented  in 
the  risk  prioritization  list. 

PM  guidance,  which  operates  as  a  tech¬ 
nique  control  function,  can  be  used,  for 
example,  to  specify  prioritization  criteria 
and  prescribe  the  format  of  the  risk 
prioritization  list. 

5.5.2. 1  Risk  Aggregation.  Figure  5-13 
shows  the  process  for  this  technique,  which 
relies  on  qualitative  judgment  and  multi¬ 
voting  methods  to  summarize  risks  at  the 
critical  risk  area  and  process  level  in  terms 
of  PF  and  CF.  The  risks  identified  in  the  RIFs 
and  Risk  Prioritization  List  are  first 


•  PM  Guidance 
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•  Program-Level  IPT 

•  Risk  Management  Coordination _ 

Figure  5-13.  Risk  Aggregation  Technique  Input  and  Output 
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grouped  according  to  critical  risk  areas  and 
processes,  and  listed  in  priority  sequence. 


Within  each  area  and  process,  the  in¬ 
dividual  risks  are  evaluated  against  a  set 
of  established  criteria  to  determine  the 
overall  aggregate  risk  rating  for  the  area / 
process.  Aggregation  criteria  needs  to  be 
established  separately  for  PF  and  CF ;  PFand 
CF  should  not  be  combined  into  a  single 
index,  e.g.,  moderate  risk.  Examples  of 
aggregation  criteria  include:  (1)  most  un¬ 
desirable  Pr  and  Cc  of  all  the  risks  within  a 
risk  area  or  process  becomes  the  aggre¬ 
gated  values  for  the  area  or  process,  or  (2) 
the  PF  and  CF  for  each  area  or  process  rep¬ 
resents  the  mean  value  for  that  area  or 
process. 


The  team  then  votes  on  each  risk  area  and 
process  to  determine  its  rating  for  PF  and 
CF,  and  the  results  are  documented.  In  ad¬ 
dition  to  the  PFand  CF  ratings  for  each  criti¬ 
cal  risk  area  and  process,  those  risks  that 
tend  to  "drive"  the  aggregate  risk  rating 
for  the  area /process  should  be  included 
in  a  list  of  aggregated  risks  to  give  sub¬ 
stance  to  the  aggregated  ratings,  e.g.,  all 
risks  in  which  either  PF  or  CF  are  rated  as 
high.  Figure  5-14  provides  a  sample  list  of 
aggregated  risks. 


PROGRAM  XY  RISK  STATUS 


Risk  Area  Status:  Design  PF:  Mi  CF:  Mi 
Significant  Design  Risks: 

1 .  Risk  Title:  Aircraft  Weight  PF:  Hi  CF:  Mi 

Problem:  Exceed  aircraft  weight  budget  by  1 0%. 
Decrease  range-payload  by  4%. 

Action:  Developing  risk-handling  plan.  User 
reviewing  requirements. 


Risk  Area  Status:  Logistics  PF:  Mi  CF:  Mod/Hi 
.  Significant  Logistics  Risks:  etc.  . 

hA/wvMA/N 

Figure  5-14.  Sample  of  a  List  of 
Prioritized  Risks 


Risk  Matrix  is  a  software  tool  that  is  de¬ 
signed  to  aid  in  managing  the  identifica¬ 
tion,  rating,  and  prioritization  of  key  risks 
that  might  affect  a  project.  It  provides  a 
structured  method  for  prioritizing  project 
risks  and  for  tracking  the  status  and  effects 
of  risk-handling  efforts.  The  major  feature 
that  Risk  Matrix  offers  the  program  office 
is  a  means  to  both  rate  and  rank  program 
risks.  This  is  helpful  in  differentiating 
among  risks  that  have  the  same  rating.  For 
example,  if  a  program  has  eight  risks  that 
the  program  office  has  evaluated/ rated  as 
high,  Risk  Matrix  provides  the  means  to 
rank  them  in  order  of  severity.  The  user  can 
use  this  ranking  as  a  guide  to  help  focus 
risk-handling  efforts.  Risk  Matrix  was  de¬ 
veloped  by  the  Air  Force  Electronic  Sys¬ 
tems  Center  (ESC)  and  The  Mitre  Corpo¬ 
ration  and  is  available  to  program  offices 
free  of  charge.  Another  useful  software  tool 
to  use  in  voting  on  risks  is  "Expert  Choice"  - 
based  on  the  Analytical  Hierarchy  Process 
(AHP).  Whatever  software  tool  is  used,  the 
analyst  should  recognize  that  a  number  of 
inherent  limitation  exist  with  such  software 
tools,  (e.g.  uninterested:  on  all  of  basing  the 
voting  process)  that  can  lead  to  erroneous 
results. 

5.6  RISK-HANDLING  TECHNIQUES 

5.6.1  General  (e.g.  Moderate  and  High 
Risk-Rated  Items) 

After  the  program's  risks  have  been  as¬ 
sessed,  the  PM  must  develop  approaches 
to  handle  significant  ones  by  analyzing 
various  handling  techniques  and  selecting 
those  best  fitted  to  the  program's  circum¬ 
stances.  The  PM  should  reflect  these  ap¬ 
proaches  in  the  program's  acquisition  strat¬ 
egy  and  include  the  specifics  on  what  is  to 
be  done  to  deal  with  the  risk,  when  it 
should  be  accomplished,  who  is  respon¬ 
sible,  and  the  cost  and  schedule  impact. 
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As  described  in  Chapter  2,  there  are  es¬ 
sentially  four  risk-handling  techniques,  or 
options.  Risk  avoidance  eliminates  the 
sources  of  high  risk  and  replaces  them 
with  a  lower-risk  solution.  Risk  transfer  is 
the  reallocation  of  risk  from  one  part  of 
the  system  to  another,  or  the  reallocation 
of  risks  between  the  Government  and  the 
prime  contractor  or  within  Government 
agencies.  Risk  control  manages  the  risk  in 
a  manner  that  reduces  the  likelihood  of 
its  occurrence  and/or  minimizes  the  risk's 
effect  on  the  program.  Risk  assumption  is 
the  acknowledgment  of  the  existence  of  a 
particular  risk  situation  and  a  conscious 
decision  to  accept  the  associated  level  of 
risk  without  engaging  in  any  special  ef¬ 
forts  to  control  it.  There  is  a  tendency  on 
many  programs  to  select  "control"  as  the 
risk-handling  option  without  seriously 
evaluating  assumption,  avoidance,  and 
transfer.  There  is  a  tendency  on  many  pro¬ 
grams  to  select  "control"  as  the  risk¬ 
handling  option  without  seriously  evalu¬ 
ating  assumptions,  avoidance,  and  trans¬ 
fer.  This  is  unwise,  since  control  may  not 
be  the  best  option,  or  even  appropriate 
option  in  some  cases.  An  unbiased  assess¬ 
ment  of  risk-handling  options  should  be 
performed  to  determine  the  most  appro¬ 
priate  option. 

In  determining  the  "best"  overall  risk¬ 
handling  strategy  and  specific  techniques 
to  be  adopted,  the  following  general  pro¬ 
cedures  apply. 

For  each  evaluated  event  risk,  all  poten¬ 
tially  applicable  techniques  should  be  iden¬ 
tified  and  evaluated,  using  the  following 
criteria: 

•  Feasibility — Feasibility  is  the  ability 
to  implement  the  handling  technique  and 
includes  an  evaluation  of  the  potential  im¬ 
pact  of  the  technique  in  the  following  areas: 


-  Technical  considerations,  such  as 
testing,  manufacturing,  and  maintainabil¬ 
ity,  caused  by  design  changes  resulting 
from  risk-handling  techniques. 

-  Adequacy  of  budget  and  schedule 
flexibility  to  apply  the  technique. 

-  Operational  issues  such  as  usability 
(man-machine  interfaces),  transportability, 
and  mobility. 

-  Organizational  and  resource  consid¬ 
erations,  e.g.,  manpower/training,  and 
structure. 

-  Environmental  issues,  such  as  the 
use  of  hazardous  materials  to  reduce  tech¬ 
nical  risk. 

-  External  considerations  beyond  the 
immediate  scope  of  the  program,  such  as 
the  impact  on  other  complementary  sys¬ 
tems  or  organizations. 

•  Cost  and  schedule  implications — 

The  risk-handling  techniques  have  a 
broad  range  of  cost  implications  in  terms 
of  dollars,  as  well  as  other  limited  re¬ 
sources,  e.g.,  critical  materials  and  na¬ 
tional  test  facilities.  The  magnitude  of  the 
cost  and  schedule  implications  will  de¬ 
pend  on  circumstances  and  can  be  as¬ 
sessed  using  such  techniques  as  cost-ben¬ 
efit  analyses  and  the  cost  and  schedule  as¬ 
sessment  techniques  previously  de¬ 
scribed.  The  approval  and  funding  of  risk¬ 
handling  techniques  should  be  part  of  the 
trade-off  process  that  establishes  and  re¬ 
fines  the  CAIV  cost  and  performance 
goals. 

•  Effect  on  the  system's  technical  per¬ 
formance — The  risk-handling  techniques 
may  affect  the  system's  capability  to 
achieve  the  required  technical  perfor¬ 
mance  objectives.  This  impact  must  be 
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clearly  understood  before  adopting  a  spe¬ 
cific  technique.  As  the  risk-handling  tech¬ 
niques  are  assessed,  the  PMO  should  at¬ 
tempt  to  identify  any  additional  param¬ 
eters  that  may  become  critical  to  technical 
performance  as  a  result  of  implementing 
them.  Trade  studies  and  sensitivity  analy¬ 
ses  can  be  useful  in  determining  the  ex¬ 
pected  effectiveness  of  this  approach. 

Once  the  risk-handling  technique  is  se¬ 
lected,  a  set  of  program  management  indi¬ 
cators  should  be  developed  to  provide 
feedback  on  program  progress,  effective¬ 
ness  of  the  risk-handling  options  selected, 
and  information  necessary  to  manage  the 
program.  These  indicators  should  consist 
of  cost  and  scheduling  data,  technical  per¬ 
formance  measures,  and  program  metrics. 

Subsequent  paragraphs  in  this  section  de¬ 
scribe  the  various  risk-handling  techniques 
cited  above. 

5.6.2  Risk  Control 

5.6.2.1  Description.  In  this  risk-handling 
technique,  the  Government  and  contractor 
take  active  steps  to  reduce  the  likelihood  of 
a  risk  event  occurring  and  to  reduce  the 
potential  impact  on  the  program.  Most  risk- 
control  steps  share  two  features:  they  require 
a  commitment  of  program  resources,  and 
they  may  require  additional  time  to  accom¬ 
plish  them.  Thus,  the  selection  of  risk-con¬ 
trol  actions  will  undoubtedly  require  some 
tradeoff  between  resources  and  the  expected 
benefit  of  the  actions.  Some  of  the  many  risk- 
control  actions  include  the  following: 

Multiple  Development  Efforts — The  use 

of  two  or  more  independent  design  teams 
(usually  two  separate  contractors,  al¬ 
though  it  could  also  be  done  internally) 
to  create  competing  systems  in  parallel  that 
meet  the  same  performance  requirements. 


Alternative  Design — Sometimes,  a  design 
option  may  include  several  risky  ap¬ 
proaches,  of  which  one  or  more  must  come 
to  fruition  to  meet  system  requirements. 
However,  if  the  PMO  studies  the  risky  ap¬ 
proaches,  it  may  be  possible  to  discover  a 
lower-risk  approach  (with  a  lower  per¬ 
formance  capability).  These  lower-risk  ap¬ 
proaches  could  be  used  as  backups  for 
those  cases  where  the  primary  ap- 
proach(es)  fail  to  mature  in  time.  This  op¬ 
tion  presumes  there  is  some  trading  room 
among  requirements.  Close  coordination 
between  the  developer  and  the  user  is  nec¬ 
essary  to  implement  lower  capability 
options. 

Trade  Studies — Systems  engineering  de¬ 
cision  analysis  methods  include  trade  stud¬ 
ies  to  solve  a  complex  design  problem.  The 
purpose  of  the  trade  studies  is  to  integrate 
and  balance  all  engineering  requirements 
in  the  design  of  a  system.  A  properly  done 
trade  study  considers  risks  associated  with 
alternatives. 

Early  Prototyping — The  nature  of  a  risk 
can  be  evaluated  by  a  prototype  of  a  sys¬ 
tem  (or  its  critical  elements)  built  and  tested 
early  in  the  system  development.  The  re¬ 
sults  of  the  prototype  can  be  factored  into 
the  design  and  manufacturing  process  re¬ 
quirements.  In  addition  to  full-up  systems, 
prototyping  is  very  useful  in  software  de¬ 
velopment  and  in  determining  a  system's 
man-machine  interface  needs.  The  key  to 
making  prototyping  successful  as  a  risk- 
control  tool  is  to  minimize  the  addition  of 
new  requirements  to  the  system  after  the 
prototype  has  been  tested  (i.e.,  requirement 
changes  not  derived  from  experience  with 
the  prototype).  Also,  the  temptation  to  use 
the  prototype  design  and  software  without 
doing  the  necessary  follow-on  design  and 
coding/ manufacturing  analyses  should  be 
avoided. 
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Incremental  Development — Incremental 
development  is  completion  of  the  system 
design  and  deployment  in  steps,  relying  on 
pre-planned  product  improvements  (P3I)  or 
software  improvements  after  the  system  is 
deployed  to  achieve  the  final  system  capa¬ 
bility.  Usually,  these  added  capabilities  are 
not  included  originally  because  of  the  high 
risk  that  they  will  not  be  ready  along  with 
the  remainder  of  the  system.  Hence,  devel¬ 
opment  is  split,  with  the  high-risk  portion 
given  more  time  to  mature.  The  basic  sys¬ 
tem,  however,  incorporates  the  provisions 
necessary  to  include  the  add-on  capabili¬ 
ties.  Incremenal  development  of  the  initial 
system  requirements  are  achieved  by  the 
basic  system. 

Technology  Maturation  Efforts — Technol¬ 
ogy  maturation  is  an  off-line  development 
effort  to  bring  an  element  of  technology  to 
the  necessary  level  so  that  it  can  be  success¬ 
fully  incorporated  into  the  system  (usually 
done  as  part  of  the  technology  transition 
process).  Normally,  technology  maturation 
is  used  when  the  desired  technology  will 
replace  an  existing  technology,  which  is 
available  for  use  in  the  system.  In  those 
cases,  technology  maturation  efforts  How¬ 
ever,  it  can  also  be  used  when  a  critical,  but 
immature,  technology  is  needed.  In  addi¬ 
tion  to  dedicated  efforts  conducted  by  the 
PMO,  Service  or  DoD-wide  technology  im¬ 
provement  programs  and  advanced  tech¬ 
nology  demonstrations  by  Government 
laboratories  as  well  as  industry  should  be 
considered. 

Robust  Design — This  approach  uses  ad¬ 
vanced  design  and  manufacturing  tech¬ 
niques  that  promote  achieving  quality 
through  design.  It  normally  results  in 
products  with  little  sensitivity  to  variations 
in  the  manufacturing  process. 

Reviews,  Walk  Throughs,  and  Inspec¬ 
tions  — These  three  risk  control  actions  can 


be  used  to  reduce  the  likelihood  and  po¬ 
tential  consequences  of  risks  through 
timely  assessments  of  actual  or  planned 
events  in  the  development  of  the  product. 
They  vary  in  the  degree  of  formality,  level 
of  participants,  and  timing. 

Reviews  are  formal  sessions  held  to  assess 
the  status  of  the  program,  the  adequacy  and 
sufficiency  of  completed  events,  and  the  in¬ 
tentions  and  consistency  of  future  events. 
Reviews  are  usually  held  at  the  completion 
of  a  program  phase,  when  significant  prod¬ 
ucts  are  available.  The  team  conducting  the 
review  should  have  a  set  of  objectives  and 
specific  issues  to  be  addressed.  The  results 
should  be  documented  in  the  form  of  ac¬ 
tion  items  to  be  implemented  by  the  PMO 
or  contractor.  The  type  of  review  will  dic¬ 
tate  the  composition  of  the  review  team, 
which  may  include  developers,  users,  man¬ 
agers,  and  outside  experts. 

A  walk  through  is  a  technique  that  can  be 
very  useful  in  assessing  the  progress  in  the 
development  of  high  or  moderate  risk 
components,  especially  software  modules. 
It  is  less  formal  than  a  review,  but  no  less 
rigorous.  The  person  responsible  for  the 
development  of  the  component  "walks 
through"  the  product  development  (to  in¬ 
clude  perceptions  of  what  is  to  be  done, 
how  it  will  be  accomplished,  and  the 
schedule)  with  a  team  of  subject-matter 
experts.  The  team  reviews  and  evaluates 
the  progress  and  plans  for  developing  the 
product  and  provides  immediate  and  less 
formal  feedback  to  the  responsible  person, 
thus  enabling  improvements  or  corrective 
actions  to  be  made  while  the  product  is 
still  under  development.  This  technique 
is  applied  during  the  development  phases, 
as  opposed  to  reviews,  which  are  normally 
held  at  the  completion  of  a  phase  or 
product. 
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Inspections  are  conducted  to  evaluate  the 
correctness  of  the  product  under  develop¬ 
ment  in  terms  of  its  design,  implementa¬ 
tion,  test  plans,  and  test  results.  They  are 
more  formal  and  rigorous  than  either  re¬ 
views  or  walk  throughs  and  are  conducted 
by  a  team  of  experts  following  a  very  fo¬ 
cused  set  of  questions  concerning  all  as¬ 
pects  of  the  product. 

Design  of  Experiments — This  is  an  engi¬ 
neering  tool  that  identifies  critical  design 
factors  that  are  difficult  to  meet. 

Open  Systems— This  approach  involves 
the  use  of  widely  accepted  commercial 
specifications  and  standards  for  selected 
system  interfaces,  products,  practices,  and 
tools.  It  provides  the  basis  for  reduced  life- 
cycle  costs,  improved  performance,  and  en¬ 
hanced  interoperability,  especially  for  long 
life  systems  with  short-life  technologies. 
Properly  selected  and  applied  commercial 
specifications  and  standards  can  result  in 
lower  risk  through  increased  design  flex¬ 
ibility;  reduced  design  time;  more  predict¬ 
able  performance;  and  easier  product  in¬ 
tegration,  support,  and  upgrade.  However, 
a  number  of  challenges  and  risks  are  asso¬ 
ciated  with  the  use  of  the  open  systems 
approach  and  must  be  considered  before 
implementation.  These  include  such  issues 
as:  maturity  and  acceptability  of  the  stan¬ 
dard,  and  its  adequacy  for  military  use;  the 
loss  of  control  over  the  development  of 
products  used  in  the  system;  the  amount 
of  product  testing  done  to  ensure  conform¬ 
ance  to  standards;  and  the  higher  configu¬ 
ration  management  workload  required. 

See  Deskbook  Section  1.2. 2.2.5  for  a  more 
detailed  discussion  of  the  use  of  open 
systems.  (Additional  information  is  also 
available  at  the  Open  Systems  Joint  Task 
Force  Website  at  www.acq.osd.mil/ 
osjtf/ .) 


Use  of  Standard  Items/Software  Reuse — 

The  use  of  standard  items  and  software 
module  reuse  should  be  emphasized  to  the 
extent  possible  to  minimize  development 
risk.  Standard  items  range  from  components 
and  assemblies  to  full-up  systems.  A  care¬ 
ful  examination  of  the  proposed  system 
option  will  often  find  more  opportunities 
for  the  use  of  standard  items  or  existing  soft¬ 
ware  modules  than  first  considered.  Even 
when  the  system  must  achieve  previously 
unprecedented  requirements,  standard 
items  can  find  uses.  A  strong  program  policy 
emphasizing  the  use  of  standard  items  and 
software  reuse  is  often  the  key  to  taking 
advantage  of  this  source  of  risk  control.  Stan¬ 
dard  items  and  software  modules  have 
proven  characteristics  that  can  reduce  risk. 
However,  the  PMO  must  be  cautious  when 
using  standard  items  in  environments  and 
applications  for  which  they  were  not  de¬ 
signed.  A  misapplied  standard  item  often 
leads  to  problems  and  failure.  Similarly,  if 
the  cycle  for  a  fielded  product  extends  for 
many  years,  it  is  possible  that  key  software 
tools  and  products  will  become  obsolete  or 
will  no  longer  be  supported.  If  this  occurs, 
costly  redesign  may  result  if  software  re¬ 
development  is  necessary. 

Two-Phase  EMD — This  risk  control  ap¬ 
proach  incorporates  a  formal  risk-reduc¬ 
tion  effort  in  the  initial  part  of  the  EMD 
phase.  It  may  involve  using  two  or  more 
contractors  with  a  down-select  occurring 
at  a  predefined  time  (normally  after  the 
preliminary  design  review).  A  logical  ex¬ 
tension  of  this  concept  is  the  "spiral"  de¬ 
velopment  model,  which  emphasizes  the 
evaluation  of  alternatives  and  risk  assess¬ 
ments  throughout  the  system's  develop¬ 
ment  and  initial  fielding. 

Use  of  Mockups — The  use  of  mockups,  es¬ 
pecially  man-machine  interface  mock-ups, 
can  be  used  to  conduct  early  exploration 
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of  design  options.  They  can  assist  in  resolv¬ 
ing  design  uncertainties  and  providing 
users  with  early  views  of  the  final  system 
configuration. 

Modeling/Simulation — The  use  of  mod¬ 
eling  and  simulation  can  provide  insights 
into  a  system's  performance  and  effective¬ 
ness  sensitivities.  Decision  makers  can  use 
performance  predictions  to  assess  a 
system's  military  worth  not  only  before  any 
physical  prototypes  are  built,  but  also 
throughout  the  system  life  cycle. 

Modeling  and  simulation  can  help  manage 
risk  by  providing  information  on  design 
capabilities  and  failure  modes  during  the 
early  stages  of  design.  This  allows  initial 
design  concepts  to  be  iterated  without  hav¬ 
ing  to  build  hardware  for  testing.  The  T&E 
community  can  use  predictive  simulations 
to  focus  the  use  of  valuable  test  assets  on 
critical  test  issues.  They  can  also  use  ex¬ 
trapolated  simulations  to  expand  the  scope 
of  evaluation  into  areas  not  readily  testable, 
thus  reducing  the  risk  of  having  the  sys¬ 
tem  fail  in  the  outer  edges  of  the  "test  en¬ 
velope."  Additionally,  a  model  can  serve 
as  a  framework  to  bridge  the  missing  pieces 
of  a  complete  system  until  those  pieces 
become  available. 

Although  modeling  and  simulation  can  be 
a  very  effective  risk-handling  tool,  it  re¬ 
quires  resources,  commitment  to  refine 
models  as  the  system  under  development 
matures,  and  a  concerted  verification  and 
validation  effort  to  ensure  that  decisions  are 
based  on  credible  information. 

Key  Parameter  Control  Boards— When  a 
particular  parameter  (such  as  system 
weight)  is  crucial  to  achieving  the  overall 
program  requirements,  a  control  board  for 
that  parameter  may  be  appropriate.  This 
board  has  representatives  from  all  affected 


technical  functions  and  may  be  chaired  by 
the  PM.  It  provides  management  focus  on 
the  parameter  and  signals  the  importance 
of  achieving  the  parameter  to  the  technical 
community.  If  staffed  properly  by  all  af¬ 
fected  disciplines,  it  can  also  help  avoid 
sacrificing  other  program  requirements  to 
achieve  that  requirement. 

Manufacturing  Screening — For  programs 
in  engineering,  manufacture,  and  devel¬ 
opment  (EMD),  various  manufacturing 
screens  (including  environmental  stress 
screening  (ESS))  can  be  incorporated  into 
test  article  production  and  low-rate  initial 
production  to  identify  deficient  manufac¬ 
turing  processes.  ESS  is  a  manufacturing 
process  for  stimulating  parts  and  work¬ 
manship  defects  in  electronic  assemblies 
and  units.  These  data  can  then  be  used  to 
develop  the  appropriate  corrective  actions. 

5.6.2.2  Procedures.  Risk  control  involves 
developing  a  risk- reduction  plan,  with  ac¬ 
tions  identified,  resourced,  and  scheduled. 
Success  criteria  for  each  of  the  risk-reduc¬ 
tion  events  should  also  be  identified.  The 
effectiveness  of  these  actions  must  be  moni¬ 
tored  using  the  types  of  techniques  de¬ 
scribed  in  Section  5.7. 

5.6.3  Risk  Avoidance 

5.6.3.1  Description.  This  technique  re¬ 
duces  risk  through  the  modification  or 
elimination  of  those  operational  require¬ 
ments,  processes  or  activities  that  cause  the 
risks.  Eliminating  operational  require¬ 
ments  requires  close  coordination  with  the 
users.  Since  this  technique  results  in  the  re¬ 
duction  of  risk,  it  should  generally  be  ini- 
tiated  in  the  development  of  a  risk¬ 
handling  plan.  It  can  be  done  in  parallel 
with  the  initial  operational  requirements 
analysis  and  should  be  supported  by  a  cost- 
benefit  analysis. 
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5.6.3.2  Procedures.  Analyzing  and  re¬ 
viewing  the  proposed  system  in  detail 
with  the  user  is  essential  to  determine  the 
drivers  for  each  operational  requirement. 
Operational  requirements  scrubbing  in¬ 
volves  eliminating  those  that  have  no 
strong  basis.  This  also  provides  the  PMO 
and  the  user  with  an  understanding  of 
what  the  real  needs  are  and  allows  them 
to  establish  accurate  system  requirements 
for  the  critical  performance.  Operational 
requirements  scrubbing  essentially  con¬ 
sists  of  developing  answers  to  the  follow¬ 
ing  questions: 

•  Why  is  the  requirement  needed? 

•  What  will  the  requirement  provide? 

•  How  will  the  capability  be  used? 

•  Are  the  requirements  specified  in 
terms  of  functions  and  capabilities,  rather 
than  a  specific  design? 

Cost/requirement  trade  studies  are  used 
to  support  operational  requirements 
scrubbing.  These  trades  examine  each  re¬ 
quirement  and  determine  the  cost  to 
achieve  various  levels  of  the  requirement 
(e.g.,  different  airspeeds,  range,  pay- 
loads).  The  results  are  then  used  to  deter¬ 
mine,  with  the  user,  whether  a  particular 
requirement  level  is  worth  the  cost  of 
achieving  that  level.  Trade  studies  are  an 
inherent  part  of  the  systems  engineering 
process.  (See  Deskbook  2.6.1  for  details  on 
systems  engineering  process.) 

5.6.4  Risk  Assumption 

5.6.4.1  Description.  This  technique  is  used 
in  every  program  and  acknowledges  the 
fact  that,  in  any  program,  risks  exist  that 
will  have  to  be  accepted  without  any  spe¬ 
cial  effort  to  control  them.  Such  risks  may 
be  either  inherent  in  the  program  or  may 


result  from  other  risk-controlling  actions 
(residual  risks).  The  fact  that  risks  are  as¬ 
sumed  does  not  mean  that  they  are  ignored. 
In  fact,  every  effort  should  be  made  to  iden¬ 
tify  and  understand  them  so  that  appro¬ 
priate  management  action  can  be  planned. 
Also,  risks  that  are  assumed  should  be 
monitored  during  development;  this  moni¬ 
toring  should  be  well-planned  from  the 
beginning. 

5.6.4.2  Procedures.  In  addition  to  the  iden¬ 
tification  of  risks  to  be  assumed,  the  fol¬ 
lowing  steps  are  key  to  successful  risk 
assumption: 

•  Identify  the  resources  (time,  money, 
people,  etc.)  needed  to  overcome  a  risk  if  it 
materializes.  This  includes  identifying  the 
specific  management  actions  that  will  be 
used,  for  example,  redesign,  retesting,  re¬ 
quirements  review,  etc. 

•  Whenever  a  risk  is  assumed,  a  sched¬ 
ule  and  cost  risk  reserve  should  be  set  aside 
to  cover  the  specific  actions  to  be  taken  if 
the  risk  occurs.  If  this  is  not  possible,  the 
program  may  proceed  within  the  funds  and 
schedule  allotted  to  the  effort.  If  the  program 
cannot  achieve  its  objectives,  a  decision  must 
be  made  to  allocate  additional  resources,  ac¬ 
cept  a  lower  level  of  capability  (lower  the 
requirements),  or  cancel  the  effort. 

•  Ensure  that  the  necessary  a  dministra- 
tive  actions  are  taken  to  quickly  report  on 
the  risk  event  and  implement  these  man¬ 
agement  actions,  such  as  contracts  for  in¬ 
dustry  expert  consultants,  arrangements 
for  test  facilities,  etc.,  and  report  on  occur¬ 
rences  of  the  risk  event. 

5.6.5  Risk  Transfer 

5. 6. 5.1  Description.  This  technique  in¬ 
volves  the  reduction  of  risk  exposure  by 
the  reallocation  of  risk  from  one  part  of  the 
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system  to  another  or  the  reallocation  of 
risks  between  the  Government  and  the 
prime  contractor,  or  between  the  prime 
contractor  and  its  sub-contractor. 

5.63.2  Procedures.  In  reallocating  risk,  de¬ 
sign  requirements  that  are  risk  drivers  are 
transferred  to  other  system  elements,  which 
may  result  in  lower  system  risk  but  still  meet 
system  requirements.  For  example,  a  high 
risk  caused  by  a  system  timing  requirement 
may  be  lowered  by  transferring  that  require¬ 
ment  from  a  software  module  to  a  specially 
designed  hardware  module  capable  of 
meeting  those  needs.  The  effectiveness  of 
requirements  reallocation  depends  on  good 
system  engineering  and  design  techniques. 
In  fact,  efficient  allocation  of  those  require¬ 
ments  that  are  risk  drivers  is  an  integral  part 
of  the  systems  engineering  process.  Modu¬ 
larity  and  functional  partitioning  are  two 
design  techniques  that  can  be  used  to  sup¬ 
port  this  type  of  risk  transfer.  In  some  cases, 
this  approach  may  be  used  to  concentrate 
risk  areas  in  one  area  of  the  system  design. 
This  allows  management  to  focus  attention 
and  resources  on  that  area. 

For  the  Government/contractor  risk-trans¬ 
fer  approach  to  be  effective,  the  risks  trans¬ 
ferred  to  the  contractor  must  be  those  that 
the  contractor  has  the  capacity  to  control 
and  manage.  These  are  generally  risks  as¬ 
sociated  with  technologies  and  processes 
used  in  the  program — those  for  which  the 
contractor  can  implement  proactive  solu¬ 
tions.  The  types  of  risks  that  are  best  man¬ 
aged  by  the  Government  include  those  re¬ 
lated  to  the  stability  of  and  external  influ¬ 
ences  on  program  requirements,  funding, 
and  schedule,  for  example.  The  contractor 
can  support  the  management  of  these  risks 
through  the  development  of  flexible  pro¬ 
gram  plans,  and  the  incorporation  of  per¬ 
formance  margins  in  the  system  and  flex¬ 
ibility  in  the  schedule.  A  number  of  options 


are  available  to  implement  risk  transfer 
from  the  Government  to  the  contractor: 
warranties,  cost  incentives,  product  perfor¬ 
mance  incentives,  and  various  types  of 
fixed  price  contracts.  A  similar  assessment 
of  prime  contractor  versus  sub-contractor 
allocation  of  risks  can  also  be  developed 
and  used  to  guide  risk  transfer  between 
these  parties. 

5.7  RISK  MONITORING 
5.7.1  General 

Risk  monitoring  is  a  continuous  process  to 
systematically  track  and  evaluate  the  per¬ 
formance  of  risk-handling  actions  against 
established  metrics  throughout  the  acqui¬ 
sition  process.  It  should  also  include  results 
of  periodic  reassessments  of  program  risk 
to  evaluate  both  known  and  new  risks  to 
the  program.  If  necessary,  the  PMO  should 
reexamine  the  risk-handling  approaches 
for  effectiveness  while  conducting  assess¬ 
ments.  As  the  program  progresses,  the 
monitoring  process  will  identify  the  need 
for  additional  risk-handling  options. 

An  effective  monitoring  effort  provides 
information  to  show  if  handling  actions  are 
not  working  and  which  risks  are  on  their 
way  to  becoming  actual  problems.  The  in¬ 
formation  should  be  available  in  sufficient 
time  for  the  PMO  to  take  corrective  action. 
The  functioning  of  IPTs  is  crucial  to  effec¬ 
tive  risk  monitoring.  They  are  the  "front 
line"  for  obtaining  indications  that  han¬ 
dling  efforts  are  achieving  their  desired 
effects. 

The  establishment  of  a  management  indi¬ 
cator  system  that  provides  accurate,  timely, 
and  relevant  risk  information  in  a  clear, 
easily  understood  manner  is  key  to  risk 
monitoring.  Early  in  the  planning  phase  of 
the  process,  PMOs  should  identify  specific 
indicators  to  be  monitored  and  information 
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to  be  collected,  compiled,  and  reported. 
Usually,  documentation  and  reporting  pro¬ 
cedures  are  developed  as  part  of  risk  man¬ 
agement  planning  before  contract  award 
and  should  use  the  contractor's  reporting 
system.  Specific  procedures  and  details  for 
risk  reporting  should  be  included  in  the 
risk  management  plans  prepared  by  the 
Government  and  the  contractor. 

To  ensure  that  significant  risks  are  effec¬ 
tively  monitored,  handling  actions  (which 
include  specific  events,  schedules,  and 
"success"  criteria)  developed  during  pre¬ 
vious  risk  management  phases  should  be 
reflected  in  integrated  program  planning 
and  scheduling.  Identifying  these  han¬ 
dling  actions  and  events  in  the  context  of 
WBS  elements  establishes  a  linkage  be¬ 
tween  them  and  specific  work  packages, 
making  it  easier  to  determine  the  impact 
of  actions  on  cost,  schedule,  and  perfor¬ 
mance.  The  detailed  information  on  risk¬ 
handling  actions  and  events  should  be 
contained  in  various  risk  management 
documentation  (both  formal  and  infor¬ 
mal).  Experience  has  shown  that  the  use 
of  an  electronic  on-line  database  that 
stores  and  permits  retrieval  of  risk-related 
information  is  almost  essential  to  effective 
risk  monitoring.  The  database  selected  or 
developed  will  depend  on  the  program. 
A  discussion  of  risk  management  informa¬ 
tion  systems  and  databases  and  suggested 
data  elements  to  be  included  in  the  data¬ 
bases  is  contained  later  in  this  chapter. 

Many  techniques  and  tools  are  available  for 
monitoring  the  effectiveness  of  risk¬ 
handling  actions,  and  PMO  personnel 
should  select  those  that  best  suit  their  needs. 
Some  monitoring  techniques  include: 

Test-Analyze-And-Fix  (TAAF) — TAAF  is 
the  use  of  a  period  of  dedicated  testing  to 
identify  and  correct  deficiencies  in  a  de¬ 


sign.  It  was  originally  conceived  as  an  ap¬ 
proach  to  improve  reliability;  it  can  also  be 
used  for  any  system  parameter  whose  de¬ 
velopment  could  benefit  from  a  dedicated 
period  of  testing  and  analysis.  Although  a 
valuable  aid  in  the  development  process, 
TAAF  should  not  be  used  in  lieu  of  a  sound 
design  process. 

Demonstration  Events — Demonstration 
events  are  points  in  the  program  (usually 
tests)  that  are  used  to  determine  if  risks 
are  being  successfully  abated.  Careful  re¬ 
view  of  the  planned  development  of  each 
risk  area  will  reveal  a  number  of  oppor¬ 
tunities  to  verify  the  effectiveness  of  the 
development  approach.  By  including  a  se¬ 
quence  of  demonstration  events  through¬ 
out  the  development,  PMO  and  contrac¬ 
tor  personnel  can  monitor  the  process  and 
identify  when  additional  efforts  are 
needed.  Demonstration  events  can  also  be 
used  as  information-gathering  actions,  as 
discussed  before,  and  as  part  of  the  risk¬ 
monitoring  process.  Table  5-2  contains  ex¬ 
amples  of  demonstration  events. 

Process  Proofing — When  particular  pro¬ 
cesses,  especially  those  of  manufacturing 
and  support,  are  critical  to  achieving 
system  requirements,  an  early  process 
proof  demonstration  is  useful  to  abate  risk. 
If  the  initial  proof  is  unsuccessful,  time  is 
still  available  to  identify  and  correct  defi¬ 
ciencies  or  to  select  an  alternative  approach. 

No  single  technique  or  tool  is  capable  of  pro¬ 
viding  a  complete  answer — a  combination 
must  be  used.  In  general,  risk  monitoring 
techniques  are  applied  to  follow  through 
on  the  planned  actions  of  the  risk-handling 
program.  They  track  and  evaluate  the  ef¬ 
fectiveness  of  handling  activities  by  com¬ 
paring  planned  actions  with  what  is  actu¬ 
ally  achieved.  These  comparisons  may  be 
as  straightforward  as  actual  versus  planned 
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ITEM 

DEMONSTRATION  EVENT 

COMPLETION  DATE 

Rocket  Motor 

Three  Case  Burst  Tests 

Propellant  Characterization 

Thermal  Barrier  Bond  Tests 

Ignition  and  Safe/Arm  Tests 

Nozzle  Assembly  Tests 

By  completion  of  preliminary  design 

1 0  Development  Motor  Firings 

—  Temperature  and  Altitude  Cycle 

—  Vibration  and  Shock 

—  Aging 

By  completion  of  final  design 

Central 

Computer 

Test  Breadboard 

Develop/Test  Unique  Microcircuits 

By  completion  of  preliminary  design 

Build/Test  Prototype 

By  completion  of  final  design 

Table  5-2.  Examples  of  Demonstration  Events 


completion  dates,  or  as  complex  as  detailed 
analysis  of  observed  data  versus  planned 
profiles.  In  any  case,  the  differences  be¬ 
tween  planned  and  actual  data  are  exam¬ 
ined  to  determine  status  and  the  need  for 
any  changes  in  the  risk-handling  approach. 

PMO  personnel  should  also  ensure  that  the 
indicators/metrics  selected  to  monitor  pro¬ 
gram  status  adequately  portray  the  true 
state  of  the  risk  events  and  handling  ac¬ 
tions.  Otherwise,  indicators  of  risks  that  are 
about  to  become  problems  will  go  unde¬ 
tected.  Subsequent  sections  identify  spe¬ 
cific  techniques  and  tools  that  will  be  use¬ 
ful  to  PMOs  in  monitoring  risks  and  pro¬ 
vide  information  on  selecting  metrics  that 
are  essential  to  the  monitoring  effort.  The 
techniques  focus  primarily  at  the  program 
level,  addressing  cost,  schedule,  and  per¬ 
formance  risks. 

5.7.2  Earned  Value  Management 

5.7.2.1  Description.  Earned  value  (EV)  is 
a  management  technique  that  relates  re¬ 
source  planning  to  schedules  and  to  tech¬ 


nical  performance  requirements.  It  is  use¬ 
ful  in  monitoring  the  effectiveness  of  risk¬ 
handling  actions  in  that  it  provides  peri¬ 
odic  comparisons  of  the  actual  work  ac¬ 
complished  in  terms  of  cost  and  schedule 
with  the  work  planned  and  budgeted. 
These  comparisons  are  made  using  a  per¬ 
formance  baseline  that  is  established  by  the 
contractor  and  the  PM  at  the  beginning  of 
the  contract  period.  This  is  accomplished 
through  the  Integrated  Baseline  Review 
(IBR)  process.  The  baseline  must  capture 
the  entire  technical  scope  of  the  program 
in  detailed  work  packages.  The  baseline 
also  includes  the  schedule  to  meet  the  re¬ 
quirements  as  well  as  the  resources  to  be 
applied  to  each  work  package.  Specific 
risk-handling  actions  should  be  included 
in  these  packages.  See  Deskbook  Section 
2.B.2.1  for  a  more  detailed  discussion  of 
Earned  Value  and  IBR. 

5.7.2.2  Procedures.  The  periodic  EV  data 
can  provide  indications  of  risk  and  the  ef¬ 
fectiveness  of  handling  actions.  When  vari¬ 
ances  in  cost  or  schedule  begin  to  appear 
in  work  packages  containing  risk-handling 
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actions,  or  in  any  work  package,  the  ap¬ 
propriate  IPTs  can  analyze  the  data  to  iso¬ 
late  causes  of  the  variances  and  gain  in¬ 
sights  into  the  need  to  modify  or  create 
handling  actions. 

5.7.3  Technical  Performance 
Measurement 

5.7.3.1  Description.  Technical  perfor¬ 
mance  measurement  (TPM)  is  a  technique 
that  compares  estimated  values  of  key  per¬ 
formance  parameters  with  achieved  values, 
and  determines  the  impact  of  any  differ¬ 
ences  on  system  effectiveness.  This  tech¬ 
nique  can  be  useful  in  risk  monitoring  by 
comparing  planned  and  achieved  values 
of  parameters  in  areas  of  known  risk.  The 
periodic  application  of  this  technique  can 
provide  early  and  continuing  predictions 
of  the  effectiveness  of  risk-handling  actions 
or  the  detection  of  new  risks  before  irrevo¬ 
cable  impacts  on  the  cost  or  schedule  occur. 

5.7.3.2  Procedures.  The  technical  perfor¬ 
mance  parameters  selected  should  be  those 
that  are  indicators  of  progress  in  the  risk¬ 
handling  action  employed.  They  can  be  re¬ 
lated  to  system  hardware,  software,  human 
factors,  and  logistics — any  product  or  func¬ 
tional  area  of  the  system.  Parameter  val¬ 
ues  to  be  achieved  through  the  planned 
handling  action  are  forecast  in  the  form  of 
planned  performance  profiles.  Achieved 
values  for  these  parameters  are  compared 
with  the  expected  values  from  the  profile, 
and  any  differences  are  analyzed  to  get  an 
indication  of  the  effectiveness  of  the 
handling  action.  For  example,  suppose  a 
system  requires  the  use  of  a  specific  tech¬ 
nology  that  is  not  yet  mature  and  the  use 
of  which  has  been  assessed  as  high  risk. 
The  handling  technique  selected  is  risk  con¬ 
trol,  and  an  off-line  technology  maturation 
effort  will  be  used  to  get  the  technology  to 
the  level  where  the  risk  is  acceptable.  The 


technology  is  analyzed  to  identify  those 
parameters  that  are  key  drivers,  and  per¬ 
formance  profiles  that  will  result  from  a 
sufficiently  mature  technology  are  estab¬ 
lished.  As  the  maturation  effort  progresses, 
the  achieved  values  of  these  parameters  are 
compared  with  the  planned  profile.  If  the 
achieved  values  meet  the  planned  profile, 
it  is  an  indicator  that  the  risk-handling  ap¬ 
proach  is  progressing  satisfactorily;  if  the 
achieved  values  fall  short  of  the  expected 
values,  it  is  an  indicator  that  the  approach 
is  failing  to  meet  expectations  and  correc¬ 
tive  action  may  be  warranted. 

5.7.4  Integrated  Planning  and 
Scheduling 

5.7.4.1  Description.  Once  a  contract  has 
been  awarded,  techniques  such  as  inte¬ 
grated  planning  and  scheduling  (inte¬ 
grated  master  plans  and  integrated  master 
schedules)  can  become  invaluable  program 
baseline  and  risk-monitoring  tools.  Inte¬ 
grated  planning  identifies  key  events,  mile¬ 
stones,  reviews,  all  integrated  technical 
tasks,  and  risk-reduction  actions  for  the 
program,  along  with  accomplishment  cri¬ 
teria  to  provide  a  definitive  measure  that 
the  required  maturity  or  progress  has  been 
achieved.  Integrated  scheduling  describes 
the  detailed  tasks  that  support  the  signifi¬ 
cant  activities  identified  in  integrated  plan¬ 
ning  and  timing  of  tasks.  Also,  the  inte¬ 
grated  schedule  can  include  the  resources 
planned  to  complete  the  tasks.  The  events, 
tasks,  and  schedule  resulting  from  inte¬ 
grated  planning  are  linked  with  contract 
specification  requirements,  WBS,  and  other 
techniques  such  as  TPM.  When  the  events 
and  tasks  are  related  to  risk-reduction 
actions,  this  linkage  provides  a  significant 
monitoring  tool,  giving  specific  insights 
into  the  relationships  among  cost,  sched¬ 
ule,  and  performance  risks. 
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5.7.4.2  Procedures.  In  integrated  plan¬ 
ning,  the  Government  and  contractor  (or 
other  performing  activity)  should  identify 
key  activities  of  the  program,  to  include 
risk-handling  actions  and  success  criteria. 
The  contractor  should  then  prepare  the  in¬ 
tegrated  schedule  reflecting  the  planned 
completion  of  tasks  associated  with  these 
activities.  As  the  program  progresses,  the 
PMO  can  monitor  effectiveness  of  han¬ 
dling  activities  included  in  the  integrated 
planning  events  and  schedule  by  compar¬ 
ing  observed  activity  results  with  their 
criteria  and  determining  any  deviations 
from  the  planned  schedule.  Any  failures 
of  handling  actions  to  meet  either  the 
event  criteria  or  schedule  should  be  ana¬ 
lyzed  to  determine  the  deviation's  impact, 
causes,  and  need  for  any  modifications  to 
the  risk-handling  approach. 


5.7.5  Watch  List 

5.7.5.1  Description.  The  watch  list  is  a 
listing  of  critical  areas  which  management 
should  pay  special  attention  to  during 
program  execution.  It  is  a  straightforward, 
easily  prepared  document  that  can  range 
in  complexity  from  a  simple  list  of  the 
identified  risks  to  one  that  includes  such 
things  as  the  priority  of  the  risk,  how  long 
it  has  been  on  the  watch  list,  the  handling 
actions,  planned  and  actual  completion 
dates  for  handling  actions,  and  explana¬ 
tions  for  any  differences.  See  Table  5-3  for 
an  example  watch  list. 

5.7.5.2  Procedures.  Watch  list  develop¬ 
ment  is  based  on  the  results  of  the  risk 
assessment.  It  is  common  to  keep  the  num¬ 
ber  of  risks  on  the  watch  list  relatively 


Potential  Risk  Area 

Risk  Reduction  Actions 

Action  Code 

Due  Date 

Date  Completed 

Explanation 

•  Accurately 

•  Use  multiple  finite 

SEA  03P31 

31  Aug  98 

predicting  shock 

element  codes  & 

environment 

simplified  numerical 

shipboard 

models  for  early 

equipment  will 

assessments. 

experience. 

•  Shock  test  simple 

SEA  03P31 

31  Aug  99 

isolated  deck,  and 
proposed  isolated 
structure  to  improve 
confidence  in 
predictions. 

•  Evaluating 

•  Concentrate  on 

SEA  03TC 

31  Aug  98 

acoustic  impact 
of  the  ship 
systems  that  are 
not  similar  to 
previous  designs. 

acoustic  modeling 
and  scale  testing  of 
technologies  not 
demonstrated 
successfully  in  large- 
scale  tests  or  full- 
scale  tests. 

•  Factor  acoustic 

SEA  03TC 

31  Aug  99 

signature  mitigation 
from  isolated  modular 

decks  into  system 
requirements. 

Continue  model  tests 

to  validate  predictions 
for  isolated  decks. 

Table  5-3.  Watch  List  Example 
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small,  focusing  on  those  that  can  have  the 
greatest  impact  on  the  program.  Items  can 
be  added  as  the  program  unfolds  and  pe¬ 
riodic  reassessments  are  conducted.  If  a 
considerable  number  of  new  risks  are  sig¬ 
nificant  enough  to  be  added  to  the  watch 
list,  it  may  be  an  indicator  that  the  original 
assessment  was  not  accurate  and  that  pro¬ 
gram  risk  is  greater  than  initially  thought. 
It  may  also  indicate  that  the  program  is  on 
the  verge  of  becoming  out  of  control.  If  a 
risk  has  been  on  the  watch  list  for  a  long 
time  because  of  a  lack  of  risk-handling 
progress,  a  reassessment  of  the  risk  or  the 
handling  approach  may  be  necessary.  Items 
on  the  watch  list  should  be  reviewed  dur¬ 
ing  the  various  program  reviews /meet¬ 
ings,  both  formal  and  informal. 

5.7.6  Reports 

5.7.6.1  Description.  Reports  are  used  to 
convey  information  to  decision  makers  and 


program  team  members  on  the  status  of 
risks  and  the  effectiveness  of  risk-handling 
actions.  Risk-related  reports  can  be  pre¬ 
sented  in  a  variety  of  ways,  ranging  from 
informal  verbal  reports  when  time  is  of  the 
essence  to  formal  summary-type  reports 
presented  at  milestone  reviews.  The  level 
of  detail  presented  will  depend  on  the 
audience. 

5.7.6.2  Procedures.  Successful  risk  manage¬ 
ment  programs  include  timely  reporting 
of  results  of  the  monitoring  process.  Re¬ 
porting  requirements  and  procedures,  to 
include  format  and  frequency,  are  normally 
developed  as  part  of  risk  management 
planning  and  are  documented  in  the  risk 
management  plan.  Reports  are  normally 
prepared  and  presented  as  part  of  routine 
program  management  activities.  They  can 
be  effectively  incorporated  into  program 
management  reviews  and  technical 


Risk 
Plan  # 


Risk  Issue 


Risk  Management  Status 

High  Moderate  Low 


94-12-9  Non-stock  Listed  Spares 


94-12-10  Engineering  Updates 


Status/Comment 

Data  still  in  review;  need  to 
assign  part  numbers. 

Data  reviewed;  updates  not 
required  at  this  time. 


94-1 2-1 1  Spares  &  Support 

94-12-12  Long  Lead  Requisitions 

94-12-13  T.O.  Validation 

94-12-14  Lack  of  LSA  Records  for 
GFE 

94-12-15  Program  Parts  Obsolescence 
94-12-51  Design  Maturity 


94-12-16  System  Y  Interface  Definition  C  3)  C  "^"3  C  D 


Spares  listing  approved  in 
definitization  conference.  No 
current  abatement  plan. 

Closed  Issue. 

Contractor  LSA  plan 
submitted  for  approval; 
rescheduled  for  5/95. 

Analysis  in  work,  identifying 
last  opportunity  buys. 

Studying  Commercial  Mix 
Interface. 

Questions  about  antenna 
location  and  cable  raised  risk. 


Figure  5-15.  Example  Showing  Detailed  List  of  Top-Level  Risk  Information 


milestones  to  indicate  any  technical,  sched¬ 
ule,  and  cost  barriers  to  the  program  ob¬ 
jectives  and  milestones  being  met.  One 
example  of  a  status  presentation  is  shown 
in  Figure  5-15.  It  shows  some  top-level  risk 
information  that  can  be  useful  to  the  PMO 
as  well  as  others  external  to  the  program. 


Although  this  level  of  reporting  can  pro¬ 
vide  quick  review  of  overall  risk  status  for 
identified  problems,  more  detailed  risk 
planning  and  status  can  be  provided  on 
individual  risk  items.  For  example,  some 
program  IPTs  have  combined  risk  level  and 
scheduled  activities  to  provide  a  graphi¬ 
cal  overview  of  risk  status  for  either  inter¬ 
nal  or  external  review.  One  method  for 
graphically  showing  risk  status  for  an  in¬ 
dividual  item  is  shown  in  Figure  5-16. 


5.7.7  Management  Indicator  System 

5.7.7.1  Description.  A  management  indi¬ 
cator  system  is  a  set  of  indicators  or  metrics 
that  provide  the  PMO  with  timely  infor¬ 
mation  on  the  status  of  the  program  and 
risk-handling  actions,  and  is  essential  to 
risk  monitoring  and  program  success.  To 
be  meaningful,  these  metrics  should  have 
some  objective  value  against  which  ob¬ 
served  data  can  be  measured,  reflecting 
trends  in  the  program  or  lack  thereof. 
Metrics  should  be  developed  jointly  by  the 
PMO  and  the  contractor.  The  contractor's 
approach  to  metrics  should  be  a  consid¬ 
eration  in  the  proposal  evaluation  process. 
If  the  contractor  does  not  have  an  estab¬ 
lished  set  of  metrics,  this  may  be  an  area 
of  risk  that  will  need  to  be  addressed. 


LACK  OF  SUPPORT  RECORDS  FOR  GFE 


^-ACTION  OPEN 
<(>- ACTION  COMPLETED 


Figure  5-16.  Example  of  More  Complex  Combination  of  Risk  Level  and  Scheduled  Tasks 
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5.7. 7. 2  Procedures.  Metrics  can  be  catego¬ 
rized  as  relating  to  technical  performance, 
cost,  and  schedule.  Technical  performance 
metrics  can  be  further  broken  down  into 
categories  such  as  engineering,  produc¬ 
tion,  and  support,  and  within  these 
groups  as  either  product-  or  process-re¬ 
lated.  Product-related  metrics  pertain  to 
characteristics  of  the  system  being  devel¬ 
oped;  they  can  include  such  things  as 
planned  and  demonstrated  values  of  the 
critical  parameters  monitored  as  part  of 
the  TPM  process  and  system-unique  data 
pertaining  to  the  different  steps  in  the 
development  and  acquisition  processes. 
Table  5-4  provides  examples  of  product- 
related  metrics. 

Process  metrics  pertain  to  the  various 
processes  used  in  the  development  and 
production  of  the  system.  For  each  pro¬ 
gram,  certain  processes  are  critical  to  the 
achievement  of  program  objectives.  Fail¬ 
ure  of  these  processes  to  achieve  their  re¬ 
quirements  is  symptomatic  of  significant 
problems.  Metrics  data  can  be  used  to  di¬ 
agnose  and  aid  in  problem  resolution. 


They  should  be  used  in  formal,  periodic 
performance  assessments  of  the  various 
development  processes  and  to  evaluate 
how  well  the  system  development  pro¬ 
cess  is  achieving  its  objectives.  DoD 
4245.7M,  Transition  from  Development 
to  Production,  and  other  supporting 
documents  such  as  NAVSO  P-6071,  Best 
Practices,  identify  seven  process  areas: 
funding,  design,  test,  production,  facili¬ 
ties,  logistics,  and  management.  Within 
each  of  these  areas,  a  number  of  specific 
processes  are  identified  as  essential  to  as¬ 
sess,  monitor,  and  establish  program  risk 
at  an  acceptable  level;  the  documents  also 
provide  risk  indicators  that  can  be  used 
as  the  basis  for  selecting  specific  process 
metrics.  Another  document.  Methods  and 
Metrics  for  Product  Success,  July  1994,  pub¬ 
lished  by  the  Office  of  the  Assistant  Sec¬ 
retary  of  the  Navy  (RD&A),  Product  In¬ 
tegrity  Directorate,  provides  a  set  of 
metrics  for  use  in  assessing  and  monitor¬ 
ing  the  design,  test,  and  production  risk 
areas.  Table  5-5  provides  examples  of 
process-related  metrics. 


Enqineering 

Requirements 

Production 

Support 

Key  Design  Parameters 
Weight 

Size 

Endurance 

Range 

Design  Maturity 

Open  problems 
reports 

Number  of  engineering 
change  proposals 

Number  of  drawings 
released 

Failure  activities 

Computer  Resource 
Utilization 

Requirements 

Traceability 

Requirements  Stability 

Manufacturing  Yields 
Incoming  Materia!  Yields 
Delinquent  Requisitions 
Unit  Production  Cost 
Process  Proofing 

Special  Tools  and  Test 
Equipment 

Support  Infrastructure 
Footprint 

Manpower  Estimates 

Table  5-4.  Examples  of  Product-Related  Metrics 
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Cost  and  schedule  metrics  can  be  used 
to  depict  how  the  program  is  progress¬ 
ing  toward  completion.  The  information 
provided  by  the  contractor  in  the  earned 
value  management  system  can  serve  as 
these  metrics,  showing  how  the  actual 
work  accomplished  compares  with  the 
work  planned  in  terms  of  schedule  and 
cost.  Other  sources  of  cost  and  schedule 
metrics  include  the  contractor's  cost 
accounting  information  and  the  integrated 
master  schedule.  Table  5-6  provides  ex¬ 
amples  of  cost  and  schedule  metrics. 


Cost 

Schedule 

Cost  variance 

Cost  performance 
index 

Estimate  at 
completion 

Management 

reserve 

Schedule  variance 

Schedule  performance 
index 

Design  schedule 
performance 

Manufacturing  schedule 
performance 

Test  schedule  performance 

Table  5-6.  Examples  of  Cost  and 
Schedule  Metrics 


Design 

Requirements 

Trade 

Studies 

Design 

Process 

Integrated  Test 
Plan 

Failure 

Reporting 

System 

Manufacturing 

Plan 

Development 
of  require¬ 
ments  trace- 
ability  plan 

Development 
of  specification 
tree 

Specifications 
reviewed  for: 

Definition  of 
all  use 

environments 

Definition  of 
all  functional 
requirements 
for  each 
mission 
performed 

Users  needs 
prioritized 

Alternative 

system 

configurations 

selected 

Test  methods 
selected 

Design  require¬ 
ments  stability 

Producibility 

analysis 

conducted 

Design  ana¬ 
lyzed  for: 

Cost 

Parts 

reduction 

Manufac¬ 

turability 

Testability 

All  develop¬ 
mental  tests  at 
system  and 
subsystem 
level  identified 

Identification  of 
who  will  to  test 
(Government, 
contractor, 
supplier) 

Contractor 
corporate-level 
management 
involved  in 
failure  report¬ 
ing  and  correc¬ 
tive  action 
process 

Responsibility 
for  analysis 
and  corrective 
action  as¬ 
signed  to 
specific 
individual  with 
close-out  date 

Plan  docu¬ 
ments  methods 
by  which 
design  to  be 
built 

Plan  contains 
sequence  and 
schedule  of 
events  at 
contractor  and 
sub-contractor 
levels  that 
defines  use  of 
materials, 
fabrication 
flow,  test 
equipment, 
tools,  facilities, 
and  personnel 

Reflects 
manufacturing 
inclusion  in 
design  pro¬ 
cess.  Includes 
identification 
and  assess¬ 
ment  of  design 
facilities 

Table  5-5.  Examples  of  Process  Metrics 
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5.8  RISK  MANAGEMENT 

INFORMATION  SYSTEMS  AND 
DOCUMENTATION 

5.8.1  Description 

To  manage  risk,  PMs  should  have  a  data¬ 
base  management  system  that  stores  and 
allows  retrieval  of  risk-related  data.  The 
risk-management  information  system  pro¬ 
vides  data  for  creating  reports  and  serves 
as  the  repository  for  all  current  and  his¬ 
torical  information  related  to  risk.  This  in¬ 
formation  may  include  risk  assessment 
documents,  contract  deliverables,  if  appro¬ 
priate,  and  any  other  risk-related  reports. 
The  PM  should  consider  a  number  of  fac¬ 
tors  in  establishing  the  management  infor¬ 
mation  system  and  developing  rules  and 
procedures  for  the  reporting  system: 

•  Assign  management  responsibility 
for  the  reporting  system 

•  Publish  any  restrictions  for  entering 
data  into  the  database 

•  Identify  reports  and  establish  a  sched¬ 
ule,  if  appropriate 

•  Use  standard  report  formats  as  much 
as  possible 


•  Ensure  that  the  standard  report  for¬ 
mats  support  all  users,  such  as  the  PM, 
IPTs,  and  IIPTs 

•  Establish  policy  concerning  access  to 
the  reporting  system  and  protect  the  data¬ 
base  from  unauthorized  access. 

With  a  well-structured  information  system, 
a  PMO  may  create  reports  for  senior  man¬ 
agement  and  retrieve  data  for  day-to-day 
program  management.  Most  likely,  the  PM 
will  choose  a  set  of  standard  reports  that 
suits  specific  needs  on  a  periodic  basis.  This 
eases  definition  of  the  contents  and  struc¬ 
ture  of  the  database.  In  addition  to  stan¬ 
dard  reports,  the  PMO  will  need  to  create 
ad  hoc  reports  in  response  to  special  que¬ 
ries,  etc.  Commercial  database  programs 
now  available  allow  the  PMO  to  create  re¬ 
ports  with  relative  ease.  Figure  5-17  shows 
a  concept  for  a  management  and  report¬ 
ing  system. 

5.8.2  Risk  Management  Reports 

The  following  are  examples  of  basic  reports 
that  a  PMO  may  use  to  manage  its  risk  pro¬ 
gram.  Each  office  should  tailor  and  amplify 
them,  if  necessary,  to  meet  specific  needs. 


Figure  5-17.  Conceptual  Risk  Management  and  Reporting  System 
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Risk  Information  Form.  The  PMO  needs 
a  document  that  serves  the  dual  purpose 
of  a  source  of  data  entry  information  and  a 
report  of  basic  information  for  the  IPTs.  The 
RIF  serves  this  purpose.  It  gives  members 
of  the  project  team,  both  Government  and 
contractors,  a  format  for  reporting  risk- 
related  information.  The  RIF  should  be 
used  when  a  potential  risk  event  is  identi¬ 
fied  and  updated  over  time  as  information 
becomes  available  and  the  status  changes. 
As  a  source  of  data  entry,  the  RIF  allows 
the  database  administrator  to  control  en¬ 
tries.  To  construct  the  database  and  ensure 
the  integrity  of  data,  the  PMO  should  de¬ 
sign  a  standard  format  for  a  RIF. 

Risk  Assessment  Report.  Risk  assessments 
form  the  basis  for  many  program  decisions, 
and  the  PM  will  probably  need  a  detailed 
report  of  any  assessment  of  a  risk  event.  A 
Risk  Assessment  Report  (RAR)  is  prepared 
by  the  team  that  assessed  a  risk  event  and 
amplifies  the  information  in  the  RIF.  It 
documents  the  identification  and  analysis 
process  and  results.  The  RAR  provides  in¬ 
formation  for  the  summary  contained  in 
the  RIF,  is  the  basis  for  developing  risk-han¬ 
dling  plans,  and  serves  as  a  historical  re¬ 
cording  of  program  risk  assessment.  Since 
RARs  may  be  large  documents,  they  may 
be  stored  as  files.  RARs  should  include  in¬ 
formation  that  links  it  to  the  appropriate 
RIF. 

Risk-Handling  Documentation.  Risk¬ 
handling  documentation  may  be  used  to 
provide  the  PM  with  the  information  he 
needs  to  choose  the  preferred  mitigation 
option  and  is  the  basis  for  the  handling  plan 
summary  that  is  contained  in  the  RIF.  This 
document  describes  the  examination  pro¬ 
cess  for  the  risk-handling  options  and  gives 
the  basis  for  the  selection  of  the 
recommended  choice.  After  the  PM  chooses 
an  option,  the  rationale  for  that  choice  may 


be  included.  There  should  be  a  plan  for 
each  risk-mitigation  task.  Risk-handling 
plans  are  based  on  results  of  the  risk  as¬ 
sessment.  This  document  should  include 
information  that  links  it  to  the  appropriate 
RIF. 

Risk  Monitoring  Documentation.  The  PM 
needs  a  summary  document  that  tracks  the 
status  oif  high  and  moderate  risks.  He  can 
produce  a  risk-tracking  list,  for  example, 
that  uses  information  that  has  been  entered 
from  the  RIF.  Each  PMO  should  tailor  the 
tracking  list  to  suit  its  needs.  If  elements  of 
needed  information  are  not  included  in  the 
RIF,  they  should  be  added  to  that  document 
to  ensure  entry  into  the  database. 

Database  Management  System  (DBMS). 

The  DBMS  that  the  PM  chooses  may  be 
commercial.  Government-owned,  or 
contractor-developed.  It  should  provide 
the  means  to  enter  and  access  data,  control 
access,  and  create  reports.  Many  options  are 
available  to  users. 

Key  to  the  MIS  are  the  data  elements  that 
reside  in  the  database.  The  items  listed  in 
Table  5-7  are  examples  of  risk  information 
that  might  be  included  in  a  database  that 
supports  risk  management.  They  are  a 
compilation  of  several  risk  reporting  forms 
used  in  current  DoD  programs  and  other 
risk  document  sources.  "Element"  is  the 
title  of  the  database  field;  "Description"  is 
a  summary  of  the  field  contents.  PMs 
should  tailor  the  list  to  suit  their  needs. 

5.9  SOFTWARE  RISK 
MANAGEMENT 
METHODOLOGIES 

The  management  of  risk  in  software  in¬ 
tensive  programs  is  essentially  the  same  as 
for  any  other  type  of  program.  A  number 
of  methodologies  specifically  focus  on  the 
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Element 

Description 

Risk  Identification 
(ID)  Number 

Identifies  the  risk  and  is  a  critical  element  of  information,  assuming  that  a 
relational  database  will  be  used  by  the  PMO.  (Construct  the  ID  number  to 
identify  the  organization  responsible  for  oversight.) 

Risk  Event 

States  the  risk  event  and  identifies  it  with  a  descriptive  name.  The  statement 
and  risk  identification  number  will  always  be  associated  in  any  report. 

Priority 

Reflects  the  importance  of  this  risk  priority  assigned  by  the  PMO  compared  to 
all  other  risks,  e.g.,  a  one  (1)  indicates  the  highest  priority. 

Data  Submitted 

Gives  the  date  that  the  RIF  was  submitted. 

Major  System/ 
Component 

Identifies  the  major  system/component  based  on  the  WBS. 

Subsystem/ 
Functional  Area 

Identifies  the  pertinent  subsystem  or  component  based  on  the  WBS. 

Category 

Identifies  the  risk  as  technical/performance  cost  or  schedule  or  combination  of 
these. 

Statement  of  Risk 

Gives  a  concise  statement  (one  or  two  sentences)  or  the  risk. 

Description  of 

Risk 

Briefly  describes  the  risk.  Lists  the  key  processes  that  are  involved  in  the 
design,  development,  and  production  of  the  particular  system  or  subsystem.  If 
technical/performance,  includes  how  it  is  manifested  (e.g.,  design  and 
engineering,  manufacturing,  etc.) 

Key 

Parameters 

Identifies  the  key  parameter,  minimum  acceptable  value,  and  goal  value,  if 
appropriate.  Identifies  associated  subsystem  values  required  to  meet  the 
minimum  acceptable  value  and  describes  the  principal  events  planned  to 
demonstrate  that  the  minimum  value  has  been  met. 

Assessment 

States  if  an  assessment  has  been  done.  Cites  the  Risk  Assessment  Report,  if 
appropriate. 

Analyses 

Briefly  describes  the  analysis  done  to  assess  the  risk.  Includes  rationale  and 
basis  for  results. 

Probability  of 
Occurrence 

States  the  likelihood  of  the  event  occurring,  based  on  definitions  in  the 
program’s  Risk  Management  Plan. 

Consequence 

States  the  consequence  of  the  event,  if  it  occurs,  based  on  definitions  in  the 
program’s  Risk  Management  Plan. 

Time  Sensitivity 

Estimates  the  relative  urgency  for  implementing  the  risk-handling  option. 

Other  Affected 

Areas 

If  appropriate,  identifies  any  other  subsystem  or  process  that  this  risk  affects. 

Risk  Handling 

Plans 

Briefly  describes  plans  to  mitigate  the  risk.  Refers  to  any  detailed  plans  that 
may  exist,  if  appropriate. 

Risk  Monitoring 
Activity 

Measures  using  metrics  for  tracking  progress  in  implementing  risk-handling 
plans  and  achieving  planned  results  for  risk  reduction. 

Status 

Briefly  reports  the  status  of  the  risk-handling  activities  and  outcomes  relevant 
to  any  risk  handling  milestones. 

Status  Due  Date 

Lists  date  of  the  status  report. 

Assignment 

Lists  individual  assigned  responsibility  for  mitigation  activities. 

Reported  By 

Records  name  and  phone  number  of  individual  who  reported  the  risk. 

Table  5-7.  Database  Management  System  Elements 
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software  aspects  of  developmental 
programs  and  can  be  useful  in  identifying 
and  analyzing  risks  associated  with  soft¬ 
ware.  Several  of  these  methodologies  are 
described  in  the  U.S.  Air  Force  publication, 
Guide  to  Software  Acquisition  and  Manage¬ 
ment.  Three  of  these  methodologies  are 
described  below. 

5.9.1  Software  Risk  Evaluation  (SRE) 

This  is  a  formal  approach  developed  by  the 
Software  Engineering  Institute  (SEI)  using 
a  risk  management  paradigm  that  defines  a 
continuous  set  of  activities  to  identify,  com¬ 
municate,  and  resolve  software  risks.  These 
activities  are  to  identify,  analyze,  plan,  track, 
and  control.  (The  SEI  activities  are  analo¬ 
gous  to  the  activities  of  the  risk  management 
process  defined  in  this  section.) 

This  methodology  is  initiated  by  the  PM, 
who  tasks  an  independent  SRE  team  to 
conduct  a  risk  evaluation  of  the  contractor's 
software  development  effort.  The  team  ex¬ 
ecutes  the  following  SRE  functions  in  per¬ 
forming  this  evaluation,  and  prepares  find¬ 
ings  that  will  provide  the  PM  with  the  re¬ 
sults  of  the  evaluation: 

•  Detection  of  the  software  technical 
risks  present  in  the  program.  An  SEI 
Taxonomy-Based  Questionnaire  is  used  to 
ensure  that  all  areas  of  potential  risk  are 
identified.  This  questionnaire  is  based  on 
the  SEI  Software  Development  Risk  Tax¬ 
onomy,  which  provides  a  systematic  way 
of  organizing  and  eliciting  risks  within  a 
logical  framework. 

•  Specification  of  all  aspects  of  identi¬ 
fied  technical  software  risks,  including 
their  conditions,  consequences,  and  source. 

•  Assessment  of  the  risks  to  determine 
the  probability  of  risk  occurrence  and  the 
severity  of  its  consequences. 


•  Consolidation  of  the  risk  data  into  a 
concise  format  suitable  for  decision  making. 

A  detailed  discussion  of  the  SRE  method¬ 
ology  is  found  in  Software  Engineering 
Institute  Technical  Report  CMU/SEI-94- 
TR-19,  Software  Risk  Evaluation  Model,  Ver¬ 
sion  1 .0,  December  1994. 

5.9.2  Boehm's  Software  Risk 
Management 

This  risk  management  methodology,  devel¬ 
oped  by  Barry  W.  Boehm  and  described  in 
IEEE  Software,  Software  Risk  Management: 
Principles  and  Practices,  January  1991,  con¬ 
sists  of  two  primary  steps,  each  with  three 
subordinate  steps.  This  risk  management 
structure  is  shown  in  Table  5-8. 

Boehm  provides  a  number  of  techniques 
that  can  be  used  to  accomplish  each  of  the 
steps  in  the  methodology.  For  example,  to 
assist  in  risk  identification,  he  includes  the 
top  10  top-level  software  risks,  based  on 
surveys  of  experienced  software  project 
managers.  These  risks  are  shown  in  Table 
5-9,  along  with  recommended  techniques 
to  manage  them.  Using  this  list  as  a  start¬ 
ing  point,  managers  and  engineers  can  then 
develop  lists  of  lower  level  risks  to  be  as¬ 
sessed  and  resolved. 

5.9.3  Best  Practices  Initiative  Risk 
Management  Method 

The  Software  Acquisition  Best  Practices 
Initiative  was  instituted  in  1994  to  improve 
and  restructure  the  software  acquisition 
management  process  through  the  identifi¬ 
cation  of  effective  practices  used  in  success¬ 
ful  software  developments.  One  result  of 
this  effort  was  the  publication  of  the  Pro¬ 
gram  Manager's  Guide  to  Software  Acquisition 
Best  Practices  by  the  Software  Program 
Managers  Network  (SPMN).  This  docu¬ 
ment  identified  nine  principal  best 
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Primary  Steps 

Secondary  Steps 

Description 

Risk  Assessment 

Risk  Identification 

-  Produces  lists  of  project  specific  risk 
events 

Risk  Analysis 

-  Assesses  probability  of  risk  event  and 
consequences 

-  Assesses  compound  risk  resulting  from 
risk  event  interaction 

Risk  Prioritization 

-  Produces  rank-ordered  list  of  identified 
and  analyzed  risk  events 

Risk  Control 

Risk  Management  Planning 

-  Produces  plan  for  addressing  each  risk 
event 

-  Integrates  individual  risk  event  plans 
with  each  other  and  the  overall  plan 

Risk  Resolution 

-  Establishes  the  environment  and 
actions  to  resolve  or  eliminate  risks 

-  Tracks  progress  in  resolving  risks 

Risk  Monitoring 

-  Provides  feedback  for  refining 
prioritization  and  plans 

Table  5-8.  Software  Risk  Management  Steps 


Risk 

Risk  Management  Techniques 

Personnel  Shortfalls 

Staffing  with  top  talent;  job  matching  team  building;  key  personnel 
agreements;  cross  training 

Unrealistic  schedules  and 
budgets 

Detailed  multisource  cost  and  schedule  estimation;  design-to-cost; 
incremental  development;  software  reuse;  requirements  scrubbing 

Developing  the  wrong  software 
functions 

Organizational  analysis;  mission  analysis;  operations  concept 
formulation;  user  surveys;  prototyping;  early  users’  manuals 

Developing  wrong  user  interface 

Task  analysis;  prototyping;  scenarios;  user  characterization 
(functionality,  style,  workload) 

Goldplating 

Requirements  scrubbing;  prototyping;  cost/benefit  analysis; 
design-to-cost 

Continuing  stream  of 
requirements  changes 

High  change  threshold;  information  hiding;  incremental 
development  (defer  changes  to  later  increments) 

Shortfalls  in  externally  furnished 
components 

Benchmarking;  inspections;  reference  checking;  compatibility 
analysis 

Shortfalls  in  internally  performed 
tasks 

Reference  checking;  pre-award  audits;  award-fee  contracts; 
competitive  design  or  prototyping;  team  building 

Real-time  performance  shortfalls 

Simulation;  benchmarking;  modeling;  prototyping;  instrumentation; 
tuning 

Straining  computer  science 
capabilities 

Technical  analysis;  cost-benefit  analysis;  prototyping;  reference 
checking 

Table  5-9.  Top  10  Software  Risks 
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Best  Practices  Initiative  Risk  Management  Method 

Address  the 
Problem 

Practice  Essentials 

Check  Status 

-  Recognize  that  all 
software  has  risk 

-  Attempt  to  resolve 
risk  as  early  as 
possible  when  cost 
impact  is  less  than 
it  will  be  later  in 
development 

-  Identify  risks 

-  Decriminalize  risk 

-  Plan  for  risk 

-  Formally  designate  a  Risk  Officer 

-  Include  in  budget  and  schedule  a  risk 
reserve  buffer  of  time,  money,  and  other 
resources 

-  Compile  database  for  all  non-negligible 
risks 

-  Prepare  profile  for  each  risk  showing 
probability  and  consequences 

-  Include  all  risks  over  full  life  cycle 

-  Provide  frequent  risk  status  reports  that 
include: 

--Top  10  risk  items 
-  Number  of  risk  items  resolved 

--  Number  of  new  risk  items 

--  Number  of  risk  items  unresolved 
--  Unresolved  risk  items  on  critical  path 

-  Probably  costs  for  unresolved  risks 

-  Risk  Officer  appointed? 

-  Risk  databases  set  up? 

-  Risk  assessments  have  clear 
impact  on  program  plans  and 
decisions? 

-  Frequency  and  timeliness  of  risk 
assessment  updates  consistent 
with  decision  updates? 

-  Objective  criteria  used  to  identify, 
assess,  and  manage  risk? 

-  Information  flow  patterns  and 
reward  criteria  support  identification 
of  risk  by  all  program  personnel? 

-  Risks  identified  throughout  entire 
lifecycle? 

-  Risk  management  reserve  exist? 

-  Risk  profile  for  every  risk,  and 
components  updated  regularly? 

-  Risk  management  plan  has  explicit 
provisions  for  altering  decision 
makers  when  risk  becomes 
imminent? 

Table  5-10.  Best  Practices  Initiative  Risk  Management  Method 


practices  that  are  essential  to  the  success 
of  any  large-scale  software  development. 
The  first  of  these  nine  is  formal  risk  man¬ 
agement.  To  assist  in  implementing  this  top 
practice,  SPMN  developed  a  three-part 
methodology  consisting  of  the  following 
steps:  address  the  problem;  practice  essen¬ 
tials;  and  check  status.  Specific  activities  as¬ 
sociated  with  these  steps  are  shown  in 
Table  5-10. 

SPMN  provides  PMOs  with  specialized 
training  programs  covering  the  core  dis¬ 
ciplines  and  techniques  for  implementing 
this  formal  risk  management  practice,  as 
well  as  the  other  best  practices.  SPMN  also 
has  available  (or  under  development)  a 
number  of  guidebooks  designed  to  pro¬ 
vide  software  developers  and  Program 


Managers  with  practical  guidance  for  plan¬ 
ning,  implementing,  and  monitoring  their 
programs. 

SPMN  can  be  contacted  at  (703)  521-5231, 
or  on  the  Internet  at  http://spmn.com/. 

In  addition  to  the  studies  by  Barry  Boehm, 
and  information  on  the  SPMN,  a  survey 
was  conducted  by  Conrow  and  Shishido 
(See  Reference)  which  evaluated  10  prior 
studies  and  categorized  the  resulting  risk 
issues  across  the  studies  into  six  categories 
and  17  total  issues,  as  shown  in  Table  5-11. 
The  very  high  degree  of  overlap  between 
risk  issues  identified  in  the  10  underlying 
studies  suggest  that  some  risk  issues  are 
common  to  many  software-intensive 
projects. 
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Risk  Grouping 

Software  Risk  Issue 

Project-Level 

1 .  Excessive,  immature,  unrealistic  or  unstable  requirements 

2.  Lack  of  involvement 

3.  Underestimation  of  project  complexity  or  dynamic  natures 

Project  Attributes 

4.  Performance  shortfalls  (includes  errors  and  quality) 

5.  Unrealistic  cost  or  schedule  (estimates  and/or  allocated  amounts) 

Management 

6.  Ineffective  project  management  (possible  at  multiple  levels) 

Engineering 

7.  Ineffective  integration,  assembly  and  test;  quality  control;  specialty 
engineering;  systems  engineering  or  (possible  at  multiple  levels) 

8.  Unanticipated  difficulties  associated  with  the  user  interface 

Work  Environment 

9.  Immature  or  untried  design,  processes  or  technologies  selected 

1 0.  Inadequate  work  plans  or  configuration  control 

11 .  Inappropriate  methods  or  tool  selection  or  inaccurate  metrics 

Other 

12.  Poor  planning 

13.  Inadequate  or  excessive  documentation  or  review  process 

14.  Legal  or  contractual  issues  (e.g.,  litigation,  malpractice,  ownership) 

15.  Obsolescence  (includes  excessive  schedule  length) 

16.  Unanticipated  difficulties  with  subcontracted  items 

1 7.  Unanticipated  maintenance  and/or  support  costs 

Table  5-11.  Software  Risk  Grouping 
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APPENDIX  A 

DoD  RISK  MANAGEMENT  POLICIES  AND  PROCEDURES 


DoD  policies  and  procedures  that  address 
risk  management  for  acquisition  programs 
are  contained  in  four  key  documents: 

•  DoD  Directive  (DoDD)  5000.1,  De¬ 
fense  Acquisition 

•  DoD  Regulation  5000. 2-R,  Man¬ 
datory  Procedures  for  Major  Defense  Ac¬ 
quisition  (MDAPs)  and  Major  Automated 
Information  System  (MAIS)  Acquisition 
Programs 

•  DoDD  5000.4,  OSD  Cost  Analysis 
Improvement  Group 

•  DoD  Manual  5000.4-M,  Cost  Analy¬ 
sis  Guidance  and  Procedures 

The  relevant  sections  of  each  document  are 
referenced  in  the  Defense  Acquisition 
Deskbook  under  Mandatory  Direction  and 
are  displayed  under  DoD- Wide  Practices. 
They  present  some  strong  statements  on 
risk  management  but  collectively  are  not 
sufficient  to  enable  the  establishment  of  an 
effective  risk  management  program.  The 
following  are  verbatim  extracts  of  sections 
of  the  DoD  5000  series  of  documents  that 
address  risk  management  as  part  of  acqui¬ 
sition  policy  and  procedures.  The  reader 
should  be  aware  that  changes  to  the  5000 
series  could  result  in  different  paragraph 
numbers.  The  paragraph  numbers  are  ac¬ 
curate  through  Change  3. 

1.  DoDD  5000.1  Defense  Acquisition, 
March  1996 

Para  D.l.a  Integrated  Management 
Framework 

"...The  acquisition  management  system 
governed  by  this  Directive  provides  for 


a  streamlined  management  process  that 
emphasizes  risk  management  and  af¬ 
fordability  and  that  explicitly  links  mile¬ 
stone  decisions  to  demonstrated  accom¬ 
plishments...  ." 

Para  D.l.d  Risk  Assessment  and 
Management 

"PMs  and  other  acquisition  managers  shall 
continually  assess  program  risks.  Risks 
must  be  well  understood,  and  risk  manage¬ 
ment  approaches  developed,  before  deci¬ 
sion  authorities  can  authorize  a  program 
to  proceed  into  the  next  phase  of  the  ac¬ 
quisition  process.  To  assess  and  manage 
risk,  PMs  and  other  acquisition  managers 
shall  use  a  variety  of  techniques,  including 
technology  demonstrations,  prototyping, 
and  test  and  evaluation.  Risk  management 
encompasses  identification,  mitigation,  and 
continuous  tracking  and  control  proce¬ 
dures  that  feed  back  through  the  program 
assessment  process  to  decision  authorities. 
To  ensure  an  equitable  and  sensible  alloca¬ 
tion  of  risk  between  government  and  in¬ 
dustry,  PMs  and  other  acquisition  manag¬ 
ers  shall  select  a  contracting  approach  ap¬ 
propriate  to  the  type  of  system  being 
acquired." 

Para  D.2.a  Event-Oriented  Management 

"The  Department  shall  use  a  rigorous, 
event-oriented  management  process  that 
emphasizes  effective  acquisition  planning, 
improved  and  continuous  communications 
with  users,  and  prudent  risk  management 
by  both  the  government  and  industry. 
Event-oriented  means  that  the  manage¬ 
ment  process  shall  be  based  on  significant 
events  in  the  acquisition  life-cycle  and  not 
arbitrary  calendar  dates." 
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Para  D.2.f  Modeling  and  Simulation 

"Models  and  simulations  shall  be  used  to 
reduce  the  time,  resources,  and  risks  of  the 
acquisition  process  and  to  increase  the 
quality  of  the  systems  being  acquired.  Rep¬ 
resentations  of  proposed  systems  (virtual 
prototypes)  shall  be  embedded  in  realistic 
synthetic  environments  to  support  the  vari¬ 
ous  phases  of  the  acquisition  process,  from 
requirements  determination  and  initial 
concept  exploration  to  the  manufacturing 
and  testing  of  new  systems,  and  related 
training." 

Pam  D.3.e  Tailoring 

"...MDAs  shall  promote  flexible,  tailored 
approaches  to  oversight  and  review  based 
on  mutual  trust  and  a  program's  size,  risk, 
and  complexity." 

2.  DoD  Regulation  5000.2-R.  Mandatory 
Procedures  for  Major  Defense  Acquisi¬ 
tion  Programs  (MDAPs)  and  Major  Au¬ 
tomated  Information  System  (MAIS)  Ac¬ 
quisition  Programs,  March  15, 1996 

Para  1.1  Purpose 

". .  .Any  singular  MDAP  or  MAIS  need  not 
follow  the  entire  process  described  below. 
However,  cognizant  of  this  model,  the  Pro¬ 
gram  Manager  (PM)  and  the  Milestone 
Decision  Authority  (MDA)  shall  structure 
the  MDAP  or  MAIS  to  ensure  a  logical  pro¬ 
gression  through  a  series  of  phases  de¬ 
signed  to  reduce  risk,  ensure  affordability, 
and  provide  adequate  information  for  de¬ 
cision  making  that  will  be  provide  the 
needed  capability  to  the  warfighter  in  the 
shortest  practical  time." 

Para  1 .2  Overview  of  the  Acquisition 
Management  Process 

"...The  acquisition  process  shall  be  struc¬ 
tured  in  logical  phases  separated  by  major 


decision  points  called  milestones.  The 
process  shall  begin  with  the  identification 
of  broadly  stated  mission  needs  that  can¬ 
not  be  satisfied  by  nonmateriel  solutions. 
Acquisition  program  stakeholders  shall 
consider  the  full  range  of  alternatives  prior 
to  deciding  to  initiate  a  new  MDAP  or 
MAIS.  Threat  projections,  system  perfor¬ 
mance,  unit  production  cost  estimates, 
life-cycle  costs,  interoperability,  cost- 
performance-schedule  trade-offs,  acquisi¬ 
tion  strategy,  affordability  constraints,  and 
risk  management  shall  be  major  consider¬ 
ations  at  each  milestone  decision  point,  in¬ 
cluding  the  decision  to  start  a  new 
program." 

". .  .At  program  initiation,  and  after  consid¬ 
eration  of  the  views  of  the  Working-Level 
Integrated  Product  Team  (IPT)  and 
Overarching  IPT  members,  the  PM  shall 
propose,  and  the  MDA  shall  consider  for 
approval,  the  appropriate  milestones,  the 
level  of  decision  for  each  milestone,  and  the 
documentation  needed  for  each  milestone. 
This  proposal  shall  consider  the  size,  com¬ 
plexity,  and  risk  of  the  program.  The  de¬ 
terminations  made  at  program  initiation 
shall  be  reexamined  at  each  milestone  in 
light  of  then-current  program  conditions." 

Para  1 .4  Acquisition  Phases  & 
Accomplishments 

"...Tailoring  shall  give  full  consideration  to 
applicable  statutes.  The  number  of  phases 
and  decision  points  shall  be  tailored  to  meet 
the  specific  needs  of  individual  PMs,  based 
on  objective  assessments  of  a  program's  cat¬ 
egory  status,  risks,  the  adequacy  of  proposed 
risk  management  plans,  and  the  urgency  of 
the  user's  need.  Tailored  acquisition  strate¬ 
gies  may  vary  the  way  in  which  core  activi¬ 
ties  are  to  be  conducted,  the  formality  of  re¬ 
views  and  documentation,  and  the  need  for 
other  supporting  activities.  ACAT II  and  III 
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program  managers  shall  work  with  their 
decision  authorities  to  tailor  any  documen¬ 
tation  and  decision  points  to  the  needs  of  the 
individual  program." 

Para  1.4.2  Phase  0:  Concept  Exploration 

. .  Phase  0  typically  consists  of  competitive, 
parallel  short-term  concept  studies.  The  fo¬ 
cus  of  these  efforts  is  to  define  and  evaluate 
the  feasibility  of  alternative  concepts  and  to 
provide  a  basis  for  assessing  the  relative  mer¬ 
its  (i.e.,  advantages  and  disadvantages,  de¬ 
gree  of  risk)  of  these  concepts  at  the  next 
milestone  decision  point. ..." 

Para  1.4.3  Phase  I:  Program  Definition 
and  Risk  Reduction 

"During  this  phase,  the  program  shall  be¬ 
come  defined  as  one  or  more  concepts,  de¬ 
sign  approaches,  and/or  parallel  technolo¬ 
gies  are  pursued  as  warranted.  Assess¬ 
ments  of  the  advantages  and  disadvantages 
of  alternative  concepts  shall  be  refined. 
Prototyping,  demonstrations,  and  early  op¬ 
erational  assessments  shall  be  considered 
and  included  as  necessary  to  reduce  risk 
so  that  technology,  manufacturing,  and 
support  risks  are  well  in  hand  before  the 
next  decision  point.  Cost  drivers,  life-cycle 
cost  estimates,  cost-performance  trades, 
interoperability,  and  acquisition  strategy  al¬ 
ternatives  shall  be  considered  to  include 
evolutionary  and  incremental  software 
development." 

Para  3.2.2.23  Cost 

"...As  the  program  progresses  through 
later  acquisition  phases,  procurement 
costs  shall  be  refined  based  on  contractor 
actual  (or  return)  costs  from  program  defi¬ 
nition  and  risk  reduction,  engineering  and 
manufacturing  development,  or  from  ini¬ 
tial  production  lots.  In  all  cases,  the  cost 
parameters  shall  reflect  the  total  program 


and  be  realistic  cost  estimates,  based  on  a 
careful  assessment  of  risks  and  realistic  ap¬ 
praisals  of  the  level  of  costs  most  likely  to 
be  realized.  The  amount  budgeted  shall 
not  exceed  the  total  cost  threshold  esti¬ 
mated  in  the  APB." 

Para  3.2.3  Exit  Criteria 

"...Exit  criteria  are  normally  selected  to 
track  progress  in  important  technical, 
schedule  or  management  risk  areas." 

Para  3.3  Acquisition  Strategy 

"...Essential  elements  in  this  context  in¬ 
clude,  but  are  not  limited  to,  open  systems, 
sources,  risk  management,  cost  as  an  inde¬ 
pendent  variable,  contract  approach,  man¬ 
agement  approach,  environmental  consid¬ 
erations,  modeling  and  simulation  ap¬ 
proach,  warranty  considerations,  and 
source  of  support.  The  PM  shall  also  ad¬ 
dress  other  major  initiatives  that  are  criti¬ 
cal  to  the  success  of  the  program. 

. . .  The  acquisition  strategy  shall  be  tailored 
to  meet  the  specific  needs  of  individual 
programs,  including  consideration  of  incre¬ 
mental  (block)  development  and  fielding 
strategies.  The  benefits  and  risk  associated 
with  reducing  lead  time  through  con¬ 
currency  shall  be  specifically  addressed  in 
tailoring  the  acquisition  strategy." 

Para  3 .3 .2 .3  Industrial  Capability 

"The  PM  shall  structure  the  acquisition 
strategy  to  promote  sufficient  program  sta¬ 
bility  to  encourage  industry  to  invest,  plan 
and  bear  risks.  Program  needs  shall  be  met 
through  reliance  on  a  national  technology 
and  industrial  base  sustained  primarily  by 
commercial  demands.  Programs  shall  mini¬ 
mize  the  need  for  new  defense-unique  in¬ 
dustrial  capabilities.  Foreign  sources  and 
international  cooperative  developments 
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shall  be  used  where  advantageous  and 
within  limitations  of  the  law  (DFARS  Part 
225). 

The  program  acquisition  strategy  shall 
analyze  the  industrial  capability  to  design, 
develop,  produce,  support  and,  if  ap¬ 
propriate,  restart  the  program.  (10  USC 
2440).  This  analysis  shall  identify  DoD 
investments  needed  to  create  new  indus¬ 
trial  capabilities,  and  the  risks  of  industry 
being  unable  to  provide  program  manufac¬ 
turing  capabilities  at  planned  cost  and 
schedule." 

Para  33.3  Cost,  Schedule,  and 
Performance  Risk  Management 

'The  PM  shall  establish  a  risk  management 
program  for  each  acquisition  program  to 
identify  and  control  performance,  cost,  and 
schedule  risks.  The  risk  management  pro¬ 
gram  shall  identify  and  track  risk  drivers, 
define  risk  abatement  plans,  and  provide 
for  continuous  risk  assessment  throughout 
each  acquisition  phase  to  determine  how 
risks  have  changed.  Risk  reduction  mea¬ 
sures  shall  be  included  in  cost-performance 
trade-offs,  where  applicable.  The  risk  man¬ 
agement  program  shall  plan  for  backups 
in  risk  areas  and  identify  design  require¬ 
ments  where  performance  increase  is  small 
relative  to  cost,  schedule,  and  performance 
risk.  The  acquisition  strategy  shall  include 
identification  of  the  risk  areas  of  the  pro¬ 
gram  and  a  discussion  of  how  the  PM  in¬ 
tends  to  manage  those  risks." 

Para  3.3.42  Cost  Management 
Incentives 

"RFPs  shall  be  structured  to  incentivize 
the  contractor  to  meet  or  exceed  cost  ob¬ 
jectives.  Whenever  applicable,  risk  reduc¬ 
tion  through  use  of  mature  processes  shall 
be  a  significant  factor  in  source  selection. 
For  industry,  competition  to  win  business. 


along  with  attendant  business  profit,  is  by 
far  the  most  powerful  incentive.  There¬ 
fore,  competition  shall  be  maintained  for 
as  long  as  practicable  in  all  acquisition 
programs." 

Para  3.3.5  Contract  Approach 

"The  acquisition  strategy  shall  discuss  the 
types  of  contracts  contemplated  for  each 
succeeding  phase,  including  considerations 
of  risk  assessment,  reasonable  risk-sharing 
by  Government  and  contractor(s),  and  the 
incentive  structure  for  contractors  to  de¬ 
crease  cost." 

Para  3. 3.5.1  Competition 

"Component  breakout  shall  be  considered 
on  every  program  and  shall  be  done  when 
there  are  significant  cost  savings  (inclusive 
of  Government  administrative  costs), 
when  the  technical  or  schedule  risk  of  fur¬ 
nishing  government  items  to  the  prime 
contractor  is  manageable,  and  when  there 
are  no  other  overriding  Governmental  in¬ 
terests  (e.g.,  industrial  capability  consid¬ 
erations  or  dependence  or  contractor  lo¬ 
gistics  support)." 

Para  33.6.6  Information  Sharing  and 
DoD  Oversight 

"DoD  oversight  activities  (i.e.,  contract  ad¬ 
ministration  offices,  contracting  offices, 
technical  activities,  and  program  manage¬ 
ment  offices)  shall  consider  all  relevant  and 
credible  information  that  might  mitigate 
risks  and  the  need  for  DoD  oversight  be¬ 
fore  designing  and  applying  direct  DoD 
oversight  of  contractor  operations." 

Para  3.4  Test  and  Evaluation 

"Test  and  evaluation  programs  shall  be 
structured  to  integrate  all  developmental 
test  and  evaluation  (DT&E),  operational 
test  and  evaluation  (OT&E),  live-fire  test 
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and  evaluation  (LFT&E),  and  modeling  and 
simulation  activities  conducted  by  differ¬ 
ent  agencies  as  an  efficient  continuum.  All 
such  activities  shall  be  part  of  a  strategy  to 
provide  information  regarding  risk  and 
risk  mitigation,  to  provide  empirical  data 
to  validate  models  and  simulations,  to  per¬ 
mit  an  assessment,  the  attainment  of  tech¬ 
nical  performance  specifications  and  sys¬ 
tem  maturity,  and  to  determine  whether 
systems  are  operationally  effective,  suit¬ 
able,  and  survivable  for  intended  use." 

Para  3.4.1  Test  and  Evaluation  Strategy 

6.  "Early  testing  of  prototypes  in  Phase  I, 
Program  Definition  and  Risk  Reduction, 
and  early  operational  assessments  shall  be 
emphasized  to  assist  in  identifying  risks." 

Para  3.4.2  Developmental  Test  and 
Evaluation 

"Developmental  test  and  evaluation  pro¬ 
grams  shall:. . . 

3.  Support  the  identification  and  descrip¬ 
tion  of  design  technical  risks;. . . 

4.  Assess  progress  toward  meeting  Criti¬ 
cal  Operational  Issues,  mitigation  of  acqui¬ 
sition  technical  risk,  achievement  of  manu¬ 
facturing  process  requirements  and  system 
maturity;...  " 

Para  3.4.3  Certification  of  Readiness  for 
Operational  Test  and  Evaluation 

"...Risk  management,  measures  and  indi¬ 
cators,  with  associated  thresholds,  which 
address  performance  and  technical  ad¬ 
equacy  of  both  hardware  and  software 
shall  be  defined  and  used  on  each 
program." 

Para  3.5.1  Life-Cycle  Cost  Estimates 
"The  life-cycle  cost  estimates  shall  be: . . . 


4.  Neither  optimistic  nor  pessimistic,  but 
based  on  a  careful  assessment  of  risks  and 
reflecting  a  realistic  appraisal  of  the  level 
of  cost  most  likely  to  be  realized." 

Para  4.3  Systems  Engineering 

". .  .The  system  engineering  process  shall  es¬ 
tablish  a  proper  balance  between  per¬ 
formance,  risk,  cost,  and  schedule,  employ¬ 
ing  a  top-down  iterative  process  of  require¬ 
ments  analysis,  functional  analysis  and 
allocation,  design  synthesis  and  verifica¬ 
tion,  and  system  analysis  and  control." 

"System  Analysis  and  Control.  System 
analysis  and  control  activities  shall  be 
established  to  serve  as  a  basis  for  evaluat¬ 
ing  and  selecting  alternatives,  measuring 
progress,  and  documenting  design  deci¬ 
sions.  This  shall  include: 

. . .  The  establishment  of  a  risk  management 
process  to  be  applied  throughout  the  de¬ 
sign  process.  The  risk  management  effort 
shall  address  the  identification  and  evalu¬ 
ation  of  potential  sources  of  technical  risks 
based  on  the  technology  being  used  and  its 
related  design,  manufacturing  capabilities, 
potential  industry  sources,  test  and  support 
processes,  risk  mitigation  efforts,  and  risk 
assessment  and  analysis.  Technology  tran¬ 
sition  planning  and  criteria  shall  be  estab¬ 
lished  as  part  of  the  overall  risk  manage¬ 
ment  effort." 

Para  4.3.5  Software  Engineering 

"Software  shall  be  managed  and  engi¬ 
neered  using  best  processes  and  practices 
that  are  known  to  reduce  cost,  schedule, 
and  performance  risks." 

Para  5.6  Cost  Analysis  Improvement 
Group  (CAIG)  Procedures 

"The  OSD  CAIG  is  established  in  ac¬ 
cordance  with  DoDD  5000.4.  The  DoD 
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Component  responsible  for  acquisition  of  a 
system  shall  work  with  the  CAIG  provid¬ 
ing  cost,  programmatic,  and  technical  infor¬ 
mation  required  to  estimate  costs  and  ap¬ 
praise  cost  risks,  and  shall  facilitate  visits  of 
the  CAIG  staff  to  the  program  office,  prod¬ 
uct  centers,  test  centers,  and  system 
contractor(s)... ." 

Para  6.2.3  Major  Automated 
Information  System  Quarterly  Report 
DD-C3I(Q)  1799 

"The  quarterly  Major  Automated  Informa¬ 
tion  System  (MAIS)  status  reporting  sys¬ 
tem  is  designed  to  provide  executive  man¬ 
agement  at  the  Component  and  OSD  lev¬ 
els  with  the  program  status,  progress, 
issues,  risks,  and  risk  reducers.  The  quar¬ 
terly  report  is  essential  to  the  early  identi¬ 
fication  of  problems  and  associated  plans 
to  initiate  corrective  actions.  The  PM  shall 
provide  the  report  to  the  MDA  in  a  timely 
manner  to  permit  prompt  action  to  ad¬ 
dress  reported  issues  and  problems...  ." 

Para  6.4  Contract  Management  Reports 

"The  reports  prescribed  by  this  section 
shall  be  used  for  all  applicable  defense 
contracts  and  are  required  for  effective 
management  of  defense  acquisitions.  Use 
of  electronic  media  shall  be  required.  The 
Work  Breakdown  Structure  (WBS)  used  in 
preparing  the  reports  covered  by  this  sec¬ 
tion  shall  be  in  conformance  with  the  pro¬ 
gram  WBS  (see  4.4.1).  Except  for  high-cost 
or  high-risk  elements,  the  normal  level  of 
reporting  detail  required  shall  be  limited 
to  level  three  of  the  contract  WBS. 

Para  6.4.1  Contractor  Cost  Data 
Reporting  (CCDR) 

"...The  following  general  policies  guide 
the  preparation  and  submission  of  CCDR 
data: 


1.  Level  of  Cost  Reporting.  Routine  re¬ 
porting  shall  be  at  the  contract  WBS  level 
three  for  prime  contractors  and  key  sub¬ 
contractors.  In  addition,  detailed  (i.e.,  sub 
level  three)  reporting  shall  be  required 
only  for  those  lower  elements  that  address 
high  risk,  high  value,  or  high  technologi¬ 
cal  interest  areas  of  a  program.  Identify¬ 
ing  these  additional  elements  is  a  critical 
early  assignment  for  program  Cost  Pro¬ 
gram-level  IPT  (which  may  include  con¬ 
tractor  membership,  where  appropriate 
and  in  accordance  with  applicable  statutes 
(see  3.3.1)).  Each  element  must  be  justified 
in  terms  of  its  contribution  to  efficient 
decision-making." 

3.  DoD  Directive  (DoDD)  5000.4,  OSD 
Cost  Analysis  Improvement  Group 
(CAIG),  November  24, 1992 

ParaD.l.h  Risk  Assessment 

"The  CAIG  Chair  report,  in  support  of  a 
milestone  review,  shall  include  quantita¬ 
tive  assessments  of  the  risk  in  the  estimate 
of  life-cycle  costs.  In  developing  an  assess¬ 
ment  of  cost  risk,  the  CAIG  shall  consider 
the  validity  of  such  programmatic  as¬ 
sumptions  of  the  CARDs  as  EMD  sched¬ 
ules,  rates  of  utilization  of  test  assets,  pro¬ 
duction  ramp  rates,  and  buy  rates,  consis¬ 
tent  with  historical  information.  The  CAIG 
shall  also  consider  uncertainties  in  inputs 
to  any  cost  estimating  relationships  used 
in  its  estimates,  as  well  as  the  uncertain¬ 
ties  inherent  in  the  calibration  of  the  CERs, 
and  shall  consider  uncertainties  in  the  fac¬ 
tors  used  in  making  any  estimates  by  anal¬ 
ogy.  The  CAIG  shall  consider  cost  and 
schedule  risk  implications  of  available  as¬ 
sessments  of  the  program's  technical  risks, 
and  may  include  the  results  in  its  cost-risk 
assessments.  The  CAIG  may  consider  in¬ 
formation  on  risk  provided  by  any  source, 
although  primary  reliance  will  be  on  the 
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technical  risk  assessments  that  are  the 
responsibility  of  the  sponsoring  DoD 
components,  and  of  other  OSD  offices,  in 
accordance  with  their  functional 
responsibilities." 

4.  DoD  5000.4-M.  Cost  Analysis  Guid¬ 
ance  and  Procedures,  December  1992 

Chapter  1:  (Outline  of  CARD 
Basic  Structure) 

Para  1.2.1.x  (..xj  Subsystem  Description 

"This  series  of  paragraphs  (repeated  for 
each  subsystem)  describes  the  major 
equipment  (hardware/software)  WBS 
components  of  the  system.  The  discussion 
should  identify  which  items  are  off-the- 
shelf.  The  technical  and  risk  issues  associ¬ 
ated  with  development  and  production  of 
individual  subsystems  also  must  be 
addressed." 

Chapter  2: 

Para  2.0  Risk 

"This  section  identifies  the  program 
manager's  assessment  of  the  program  and 
the  measures  being  taken  or  planned  to 
reduce  those  risks.  Relevant  sources  of  risk 
include:  design  concept,  technology  devel¬ 
opment,  test  requirements,  schedule,  acqui¬ 
sition  strategy,  funding  availability,  con¬ 
tract  stability,  or  any  other  aspect  that  might 
cause  a  significant  deviation  from  the 
planned  program.  Any  related  external 


technology  programs  (planned  or  on¬ 
going)  should  be  identified,  their  potential 
contribution  to  the  program  described,  and 
their  funding  prospects  and  potential  for 
success  assessed.  This  section  should  iden¬ 
tify  these  risks  for  each  acquisition  phase 
(DEM/VAL,  EMD,  productions  and  de¬ 
ployment,  and  O&S)." 

Para  2. B. 9  Sensitivity  Analysis 

"The  sensitivity  of  projected  costs  to  criti¬ 
cal  program  assumptions  shall  be  exam¬ 
ined.  Aspects  of  the  program  to  be 
subjected  to  sensitivity  analysis  shall  be 
identified  in  the  DoD  CCA  of  program  as¬ 
sumptions.  The  analysis  shall  include  fac¬ 
tors  such  as  learning  curve  assumptions; 
technical  risk,  i.e.,  the  risk  of  more  devel¬ 
opment  and/or  production  effort,  changes 
in  performance  characteristics,  schedule  al¬ 
terations,  and  variations  in  testing  require¬ 
ments;  and  acquisition  strategy  (multiyear 
procurement,  dual  sourcing,  etc.)." 

Para  2.C.3  PM  Presentation 

"The  Program  Manager's  designated  repre¬ 
sentative  shall  present  the  CAIG  with  the 
POE  for  each  alternative  under  construction 
and  explain  how  each  is  derived.  This  pre¬ 
sentation  shall  cover  the  estimates  and  esti¬ 
mating  procedures  at  the  major  subcompo¬ 
nent  level  (e.g.,  airframe,  engine,  major  avi¬ 
onics  subsystem,  etc.).  The  presentation 
should  focus  on  the  items  that  are  cost  drivers 
and/or  elements  of  high  cost  risk." 


A -7 


APPENDIX  B 

GENERIC  RISK  MANAGEMENT  PLAN 


SAMPLE  RISK  MANAGEMENT  PLAN 
Preface 

DoDD  5000.1  requires  that  "PMs  and  other  acquisition  managers  shall  continually  assess 
program  risks"  and  that  they  "shall  develop  a  contracting  approach  appropriate  to  the 
type  of  system  being  acquired."  Further,  DoD  5000.2-R  states  that  for  ACAT I  Programs, 
"The  PM  shall  establish  a  risk  management  program. . .  to  identify  and  control  performance, 
cost,  and  schedule  risks."  Although  the  need  for  a  risk  management  program  and  a  risk 
management  process  are  addressed  throughout  this  regulation,  there  is  no  requirement 
for  a  formal  Risk  Management  Plan  (RMP).  However,  Program  Managers  (PMs)  have 
found  such  a  plan  necessary  to  focus  properly  on  the  assessment  and  handling  of  pro¬ 
gram  risk,  a  core  acquisition  management  issue  that  Milestone  Decision  Authorities  (MDAs) 
must  rigorously  address  at  appropriate  milestones  before  making  program  decisions. 

Attached  is  a  sample  format  for  a  RMP  that  is  a  compilation  of  several  good  risk  plans  and 
the  results  of  the  DoD  Risk  Management  Working  Group  Study.  It  represents  the  types  of 
information  and  considerations  that  a  plan,  tailored  to  a  specific  program,  might  contain. 
There  are  also  two  example  of  Risk  Management  Plans — one  for  an  ACAT  I  or  II  Program, 
the  other  for  an  ACAT  III  or  IV  Program.  The  DoD  Acquisition  Deskbook,  Section  2.5.2,  has 
general  guidance  and  advice  in  all  areas  of  risk  management.  Section  2.5.2.4  of  the  Deskbook 
contains  information  concerning  the  development  of  a  risk  management  plan.  The  infor¬ 
mation  in  this  Guide  is  consistent  with,  and  in  most  cares  identical  to,  the  Deskbook. 

There  is  a  danger  in  providing  a  sample  document.  First  of  all,  because  it  is  written  as  a 
guide  for  a  general  audience,  it  does  not  satisfy  all  of  the  needs  of  any  particular  program. 
Second,  there  is  the  possibility  that  some  prospective  user  will  simply  adopt  the  plan  as 
written,  despite  the  fact  that  it  does  not  fit  his  or  her  program.  We  discourage  this. 

The  reason  for  providing  this  sample  format  is  to  give  PMs  and  their  staffs  a  starting  point 
for  their  own  planning  process.  It  should  stimulate  thought  about  what  has  to  be  done 
and  give  some  ideas  on  how  to  begin  writing  a  plan.  The  sample  plan  contains  more  informa¬ 
tion  than  most  program  offices  should  need.  Few  PMs  have  the  resources  for  a  dedicated  risk 
management  effort  as  depicted  in  the  plan.  The  key  to  using  the  sample  plan  is  to  keep 
things  simple  and  tailor  the  plan  to  suit  your  needs,  focusing  on  the  management  of  risk  in  the 
key  critical  areas  of  your  program. 

The  italicized  text  reflects  the  outline  of  a  risk  management  plan  found  in  the  DoD  Acquisition 
Deskbook  section  2.52.4,  Figure  2.52.4-2. 
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SAMPLE  FORMAT  FOR  RISK  MANAGEMENT  PLAN 

INTRODUCTION.  This  section  should  address  the  purpose  and  objective  of  the  plan,  and  provide 
a  brief  summary  of  the  program,  to  include  the  approach  being  used  to  manage  the  program,  and  the 
acquisition  strategy. 

PROGRAM  SUMMARY.  This  section  contains  a  brief  description  of  the  program,  including  the 
acquisition  strategy  and  the  program  management  approach.  The  acquisition  strategy  should  ad¬ 
dress  its  linkage  to  the  risk  management  strategy. 

DEFINITIONS.  Definitions  used  by  the  program  office  should  be  consistent  with  DoD  definitions 
for  ease  of  understanding  and  consistency.  However,  the  DoD  definitions  allow  program  managers 
flexibility  in  constructing  their  risk  management  programs.  Therefore,  each  program's  risk  man¬ 
agement  plan  may  include  definitions  that  expand  the  DoD  definitions  to  fit  its  particular  needs. 
For  example,  each  plan  should  include,  among  other  things,  definitions  for  the  ratings  used  for 
technical,  schedule  and  cost  risk.  (Discussion  of  risk  rating  is  contained  in  Acquisition  Deskbook 
Section  2. 5. 2.1.) 

RISK  MANAGEMENT  STRATEGY  AND  APPROACH.  Provide  an  overview  of  the  risk  man¬ 
agement  approach,  to  include  the  status  of  the  risk  management  effort  to  date,  and  a  description  of 
the  program  risk  management  strategy.  See  Acquisition  Deskbook  Sections  2.5.2.1  and  2.5.23. 

ORGANIZATION.  Describe  the  risk  management  organization  of  the  program  office  and  list  the 
responsibilities  of  each  of  the  risk  management  participants.  See  Acquisition  Deskbook  Section 
2.5.23. 

RISK  MANAGEMENT  PROCESS  AND  PROCEDURES.  Describe  the  program  risk  manage¬ 
ment  process  to  be  employed;  i.e.,  risk  planning,  assessment,  handling,  monitoring  and  documen¬ 
tation,  and  a  basic  explanation  of  these  components.  See  Acquisition  Deskbook  Section  2521.  Also 
provide  application  guidance  for  each  of  the  risk  management  functions  in  the  process.  If  possible, 
the  guidance  should  be  as  general  as  possible  to  allow  the  program's  risk  management  organization 
(e.g.,  IPTs)  flexibility  in  managing  the  program  risk,  yet  specific  enough  to  ensure  a  common  and 
coordinated  approach  to  risk  management.  It  should  address  how  the  information  associated  with 
each  element  of  the  risk  management  process  will  be  documented  and  made  available  to  all  partici¬ 
pants  in  the  process,  and  how  risks  will  be  tracked,  to  include  the  identification  of  specific  metrics  if 
possible. 

RISK  PLANNING.  This  section  describes  the  risk  planning  process  and  provides  guidance  on 
how  it  will  be  accomplished,  and  the  relationship  between  continuous  risk  planning  and  this  RMP. 
Guidance  on  updates  of  the  RMP  and  the  approval  process  to  be  followed  should  also  be  included. 
See  Section  2.5. 2.1  of  the  Deskbook  for  information  on  risk  planning. 

RISK  ASSESSMENT.  This  section  of  the  plan  describes  the  assessment  process  and  procedures  for 
examining  the  critical  risk  areas  and  processes  to  identify  and  document  the  associated  risks.  It  also 
summarizes  the  analyses  process  for  each  of  the  risk  areas  leading  to  the  determination  of  a  risk 
rating.  This  rating  is  a  reflection  of  the  potential  impact  of  the  risk  in  terms  of  its  variance  from 
known  Best  Practices  or  probability  of  occurrence,  its  consequence,  and  its  relationship  to  other  risk 
areas  or  processes.  This  section  may  include: 
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•  Overview  and  scope  of  the  assessment  process 

•  Sources  of  information 

•  Information  to  be  reported  and  formats 

•  Description  of  how  risk  information  is  documented 

•  Assessment  techniques  and  tools  (see  Section  2.52.4  of  the  Deskbook) 

RISK  HANDLING.  This  section  describes  the  procedures  that  can  be  used  to  determine  and  evaluate 
various  risk  handling  options,  and  identifies  tools  that  can  assist  in  implementing  the  risk  han¬ 
dling  process.  It  also  provides  guidance  on  the  use  of  the  various  handling  options  for  specific  risks. 

RISK  MONITORING.  This  section  describes  the  process  and  procedures  that  will  be  followed  to 
monitor  the  status  of  the  various  risk  events  identified.  It  should  provide  criteria  for  the  selection  of 
risks  to  be  reported  on,  and  the  frequency  of  reporting.  Guidance  on  the  selection  of  metrics  should 
also  be  included. 

RISK  MANAGEMENT  INFORMATION  SYSTEM,  DOCUMENTATION  AND  REPORTS. 

This  section  describes  the  MIS  structure,  rules,  and  procedures  that  will  be  used  to  document  the 
results  of  the  risk  management  process.  It  also  identifies  the  risk  management  documentation  and 
reports  that  will  be  prepared;  specifies  the  format  and  frequency  of  the  reports;  and  assigns  respon¬ 
sibility  for  their  preparation . 
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SAMPLE  RISK  MANAGEMENT  PLAN  FOR  THE  XYZ  PROGRAM  (ACAT  I,  II) 
1.0  INTRODUCTION 


1.1  PURPOSE 

This  Risk  Management  Plan  (RMP)  presents  the  process  for  implementing  proactive  risk 
management  as  part  of  the  overall  management  of  the  XYZ  program.  Risk  management  is 
a  program  management  tool  to  assess  and  mitigate  events  that  might  adversely  impact 
the  program.  Therefore,  risk  management  increases  the  likelihood  of  program  success. 
This  RMP  will: 

•  Serve  as  a  basis  for  identifying  alternatives  to  achieve  cost,  schedule,  and  perfor¬ 
mance  goals, 

•  Assist  in  making  decisions  on  budget  and  funding  priorities, 

•  Provide  risk  information  for  Milestone  decisions,  and 

•  Allow  monitoring  the  health  of  the  program  as  it  proceeds. 

The  RMP  describes  methods  for  identifying,  analyzing,  prioritizing,  and  tracking  risk 
drivers;  developing  risk-handling  plans;  and  planning  for  adequate  resources  to  handle 
risk.  It  assigns  specific  responsibilities  for  the  management  of  risk  and  prescribes  the 
documenting,  monitoring,  and  reporting  processes  to  be  followed. 

This  is  the  second  edition  of  the  Risk  Management  Plan  for  the  XYZ  program.  The  initial 
plan  concentrated  on  tasks  leading  to  Milestone  I  (Phase  0);  this  plan  concentrates  on 
the  tasks  leading  to  Milestone  II  (Phase  I).  Subsequent  updates  to  this  RMP  will  shift 
focus  to  the  later  acquisition  phases.  There  are  changes  in  every  area  of  the  plan;  they 
include  refinement  of  the  risk  identification  process.  The  PMO  Risk  Management  Coor¬ 
dinator  has  been  identified  and  training  of  IPT  members  has  commenced. 

1.2  PROGRAM  SUMMARY 

The  XYZ  program  was  initiated  in  response  to  Mission  Need  Statement  (MNS)  XXX, 
dated  DD-MM-YYYY  and  Operational  Requirements  Document  (ORD),  dated  DD-MM- 
YYYY.  It  is  required  to  support  the  fundamental  objective  of  U.S.  defense  policy  as  stated 
in  Defense  Planning  Guidance  (DPG)  and  the  National  Military  Strategy.  The  XYZ  sys¬ 
tem  is  based  on  the  need  for  an  integrated  combat  system  to  link  battlefield  decision 
makers.  The  XYZ  mission  areas  are:  (Delineate  applicable  areas). 

The  XYZ  program  will  develop  and  procure  120  advanced  platforms  to  replace  the  ag¬ 
ing  ABC  platforms  currently  in  the  inventory.  In  order  to  meet  force  structure  objec¬ 
tives,  the  XYZ  system  must  reach  Initial  Operational  Capability  (IOC)  (four  platforms) 
by  FY-07.  The  program  is  commencing  an  eight-year  EMD  phase  that  will  be  followed 
by  a  five-year  procurement  phase.  The  objectives  of  the  EMD  phase  are  to  (discuss  the 
specific  objectives  of  this  phase).  The  program  has  Congressional  interest  and  is  restricted 
to  a  Research  and  Development  funding  ceiling  of  $300  million. 
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1.2.1  System  Description 

The  XYZ  will  be  an  affordable,  yet  capable,  platform  taking  advantage  of  technological 
simplification  and  advancements.  The  XYZ  integrated  Combat  System  includes  all  non¬ 
propulsion  electronics  and  weapons.  Subsystems  provide  capabilities  in  combat  control, 
electronic  warfare  support  measures  (ESM),  defensive  warfare,  navigation,  radar,  interior 
communications,  monitoring,  data  transfer,  tactical  support  device,  exterior  communica¬ 
tions,  and  Identification  Friend  or  Foe  (IFF).  Weapons  systems  are  to  be  provided  by  the 
program  offices  that  are  responsible  for  their  development.  The  Mechanical  and  Electrical 
(M&E)  system  comprises....  The  Combat  System,  M&E  systems,  and  subsystems  provide 
the  XYZ  system  with  the  capability  and  connectivity  to  accomplish  the  broad  range  of 
missions  defined  in  the  MNS  and  ORD. 

1.2.2  Acquisition  Strategy 

The  XYZ  program  initial  strategy  is  to  contract  with  one  prime  contractor  in  Program 
Definition/Risk  Reduction  (PDRR)  for  development  of  two  prototype  systems  for  test 
and  design  validation.  Due  to  the  technical  complexity  of  achieving  the  performance  lev¬ 
els  of  the  power  generation  systems,  the  prime  will  use  two  sub-contractors  for  the  engine 
development  and  down  select  to  one  producer  prior  to  low  rate  initial  production,  which 
is  scheduled  for  FY-04.  Various  organizations,  such  as  the  Government  Research  Labora¬ 
tory  will  be  funded  to  provide  experts  for  assessment  of  specific  areas  of  risk.  The  pro¬ 
gram  has  exit  criteria,  included  in  the  list  of  Critical  Program  Attributes  in  Annex  A,  that 
must  be  met  before  progressing  to  the  next  phase. 

1.2.3  Program  Management  Approach 

The  XYZ  program  is  managed  using  the  IPPD  concept,  with  program  integrated  product 
teams  (PIPTs)  established  largely  along  the  hierarchy  of  the  product  work  breakdown 
structure  (WBS).  There  are  also  cost-performance  and  test  Working  IPTs  (WIPTs)  estab¬ 
lished  for  vertical  coordination  up  the  chain  of  command.  The  PM  chairs  a  program  inte¬ 
grating  IPT  (IIPT)  that  addresses  issues  that  are  not  resolved  at  the  WIPT  level. 

1.3  DEFINITIONS 

1.3.1  Risk 

Risk  is  a  measure  of  the  inability  to  achieve  overall  program  objectives  within  defined 
cost,  schedule,  and  technical  constraints  and  has  two  components:  (1)  the  probability  of 
failing  to  achieve  a  particular  outcome  and  (2)  the  consequences  of  failing  to  achieve  that 
outcome.  For  processes,  risk  is  a  measure  of  the  difference  between  actual  performance  of 
a  process  and  the  known  best  practice  for  performing  that  process. 

1.3.2  Risk  Event 

Risk  events  are  those  events  within  the  XYZ  program  that,  if  they  go  wrong,  could  result 
in  problems  in  the  development,  production,  and  fielding  of  the  system.  Risk  events  should 
be  defined  to  a  level  such  that  the  risk  and  causes  are  understandable  and  can  be  accurately 
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assessed  in  terms  of  likelihood/ probability  and  consequence  to  establish  the  level  of  risk. 
For  processes,  risk  events  are  assessed  in  terms  of  process  variance  from  known  best  prac¬ 
tices  and  potential  consequences  of  the  variance. 

1.3.3  Technical  Risk 

This  is  the  risk  associated  with  the  evolution  of  the  design  and  the  production  of  the  XYZ 
system  affecting  the  level  of  performance  necessary  to  meet  the  operational  requirements. 
The  contractor's  and  subcontractors'  design,  test,  and  production  processes  (process  risk) 
influence  the  technical  risk  and  the  nature  of  the  product  as  depicted  in  the  various  levels 
of  the  Work  Breakdown  Structure  (product  risk). 

1.3.4  Cost  Risk 

This  is  the  risk  associated  with  the  ability  of  the  program  to  achieve  its  life-cycle  cost 
objectives.  Two  risk  areas  bearing  on  cost  are  (1)  the  risk  that  the  cost  estimates  and  objec¬ 
tives  are  accurate  and  reasonable  and  (2)  the  risk  that  program  execution  will  not  meet  the 
cost  objectives  as  a  result  of  a  failure  to  handle  cost,  schedule,  and  performance  risks. 

1.3.5  Schedule  Risk 

These  risks  are  those  associated  with  the  adequacy  of  the  time  estimated  and  allocated  for 
the  development,  production,  and  fielding  of  the  system.  Two  risk  areas  bearing  on  schedule 
risk  are  (1)  the  risk  that  the  schedule  estimates  and  objectives  are  realistic  and  reasonable 
and  (2)  the  risk  that  program  execution  will  fall  short  of  the  schedule  objectives  as  a  result 
of  failure  to  handle  cost,  schedule,  or  performance  risks. 

1.3.6  Risk  Ratings 

This  is  the  value  that  is  given  to  a  risk  event  (or  the  program  overall)  based  on  the  analysis 
of  the  likelihood /probability  and  consequences  of  the  event.  For  the  XYZ  program,  risk 
ratings  of  Low,  Moderate,  or  High  will  be  assigned  based  on  the  following  criteria.  See 
Section  3.3.2  of  this  appendix  for  guidance  on  determining  likelihood  and  consequences. 
When  rating  process  variance  from  best  practices,  there  is  no  rating  of  likelihood /prob¬ 
ability,  rather  the  level  would  be  a  measure  of  the  variance  from  best  practices  (see  Para¬ 
graph  3.3.2.3). 

•  Low  Risk:  Has  little  or  no  potential  for  increase  in  cost,  disruption  of  schedule,  or 
degradation  of  performance.  Actions  within  the  scope  of  the  planned  program  and  nor¬ 
mal  management  attention  should  result  in  controlling  acceptable  risk. 

•  Moderate  Risk:  May  cause  some  increase  in  cost,  disruption  of  schedule,  or  degra¬ 
dation  of  performance.  Special  action  and  management  attention  may  be  required  to  handle 
risk. 

•  High  Risk:  Likely  to  cause  significant  increase  in  cost,  disruption  of  schedule,  or 
degradation  of  performance.  Significant  additional  action  and  high  priority  management 
attention  will  be  required  to  handle  risk. 
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1.3.7  Independent  Risk  Assessor 

An  independent  risk  assessor  is  a  person  who  is  not  in  the  management  chain  or  directly 
involved  in  performing  the  tasks  being  assessed.  Use  of  independent  risk  assessors  is  a  valid 
technique  to  ensure  that  all  risk  areas  are  identified  and  that  the  consequence  and  likelihood/ 
probability  (or  process  variance)  are  properly  understood.  The  technique  can  be  used  at  differ¬ 
ent  program  levels,  e.g..  Program  Office,  Service  Field  Activities,  Contractors,  etc.  The  Pro¬ 
gram  Manager  will  approve  the  use  of  independent  assessors,  as  needed. 

1.3.8  Templates  and  Best  Practices 

A  "template"  is  a  disciplined  approach  for  the  application  of  critical  engineering  and  manu¬ 
facturing  processes  that  are  essential  to  the  success  of  most  programs.  DoD  4245.7-M,  Transi¬ 
tion  from  Development  to  Production  Solving  the  Risk  Equation,  provides  a  number  of  such  tem¬ 
plates.  For  each  template  process  described  in  DoD  4245.7-M,  a  Best  Practice  Information  is 
described  in  NAVSO  P-6071 .  These  documents  outline  the  ideal  or  low  risk  approach  and  thus 
serve  as  a  baseline  from  which  risk  for  some  XYZ  processes  can  be  assessed. 

1.3.9  Metrics 

There  are  measures  used  to  indicate  progress  or  achievement. 

1.3.10  Critical  Program  Attributes 

Critical  Program  Attributes  are  performance,  cost,  and  schedule  properties  or  values  that  are 
vital  to  the  success  of  the  program.  They  are  derived  from  various  sources,  such  as  the  Acqui¬ 
sition  Program  Baseline,  exit  criteria  for  the  next  program  phase.  Key  Performance  Param¬ 
eters,  test  plans,  the  judgment  of  program  experts,  etc.  The  XYZ  program  will  track  these 
attributes  to  determine  the  progress  in  achieving  the  final  required  value.  See  Annex  A  for  a 
list  of  the  XYZ  Critical  Program  Attributes. 

2.0  RISK  MANAGEMENT  APPROACH 

2.1  GENERAL  APPROACH  AND  STATUS 

DoD  Directive  5000.1  states:  "Risks  must  be  well  understood,  and  risk  management  ap¬ 
proaches  developed/before  decision  authorities  can  authorize  a  program  to  proceed  into 
the  next  phase  of  the  acquisition  process."  This  policy  is  implemented  in  DoD  Regulation 
5000.2-R,  with  more  detailed  guidance  provided  in  the  individual  Service  regulation.  The 
Defense  Acquisition  Deskbook  (Section  2.5.2)  provides  additional  guidance,  advice,  and  wis¬ 
dom  on  the  management  of  risk.  Figure  2-1  shows  how  the  XYZ  program  risk  manage¬ 
ment  fits  into  the  phases  and  milestones  of  the  acquisition  process. 

The  XYZ  program  will  use  a  centrally  developed  risk  management  strategy  throughout 
the  acquisition  process  and  decentralized  risk  planning,  assessment,  handling,  and  moni¬ 
toring.  XYZ  risk  management  is  applicable  to  all  acquisition  functional  areas. 

The  results  of  the  Concept  Exploration  Phase  of  the  program  identified  potential  risk  events 
and  the  Acquisition  Strategy  reflects  the  program's  risk-handling  approach.  Overall,  the 
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Figure  2-1.  Risk  Management  and  the  Acquisition  Process 


risk  of  the  XYZ  program  for  Milestone  I  was  assessed  as  moderate,  but  acceptable.  Moder¬ 
ate  risk  functional  areas  were  threat,  manufacturing,  cost,  funding,  and  schedule.  The 
remaining  functional  areas  of  technology,  design  and  engineering  (hardware  and  soft¬ 
ware),  support,  (schedule)  concurrency,  human  systems  integration,  and  environmental 
impact  were  assessed  as  low  risk. 


2.2  RISK  MANAGEMENT  STRATEGY 

The  basic  risk  management  strategy  is  intended  to  identify  critical  areas  and  risk  events, 
both  technical  and  non-technical,  and  take  necessary  action  to  handle  them  before  they 
can  become  problems,  causing  serious  cost,  schedule,  or  performance  impacts.  This  pro¬ 
gram  will  make  extensive  use  of  modeling  and  simulation,  technology  demonstrations, 
and  prototype  testing  in  handling  risk. 

Risk  management  will  be  accomplished  using  the  integrated  Government-Contractor  IPT 
organization.  These  IPTs  will  use  a  structured  assessment  approach  to  identify  and  ana¬ 
lyze  those  processes  and  products  that  are  critical  to  meeting  the  program  objectives.  They 
will  then  develop  risk-handling  options  to  mitigate  the  risks  and  monitor  the  effective¬ 
ness  of  the  selected  handling  options.  Key  to  the  success  of  the  risk  management  effort  is 
the  identification  of  the  resources  required  to  implement  the  developed  risk-handling 
options. 
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Risk  information  will  be  captured  by  the  IPTs  in  a  risk  management  information  system 
(RMIS)  using  a  standard  Risk  Information  Form  (RIF).  The  RMIS  will  provide  standard 
reports,  and  is  capable  of  preparing  ad  hoc  tailored  reports.  See  Annex  B  for  a  description 
of  the  RMIS  and  RIF. 

Risk  information  will  be  included  in  all  program  reviews,  and  as  new  information  be¬ 
comes  available,  the  PMO  and  contractor  will  conduct  additional  reviews  to  ascertain  if 
new  risks  exist.  The  goal  is  to  be  continuously  looking  to  the  future  for  areas  that  may 
severely  impact  the  program. 

2.3  ORGANIZATION 

The  risk  organization  for  the  XYZ  program  is  shown  in  Figure  2-2.  This  is  not  a  separate 
organization,  but  rather  shows  how  risk  is  integrated  into  the  program's  existing  organi¬ 
zation  and  shows  risk  relationships  among  members  of  the  program  team. 

2.3.1  Risk  Management  Coordinator 

The  Risk  Management  Coordinator,  the  XYZ  Technology  Assessment  and  R&D  Manager, 
is  overall  coordinator  of  the  Risk  Management  Program.  The  Risk  Management  Coordi¬ 
nator  is  responsible  for: 

•  Maintaining  this  Risk  Management  Plan 

•  Maintaining  the  Risk  Management  Database 


•  Briefing  the  PM  on  the  status  of  XYZ  program  risk 


Figure  2-2.  XYZ  Risk  Management  Organization 
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•  Tracking  efforts  to  reduce  moderate  and  high  risk  to  acceptable  levels 

•  Providing  risk  management  training 

•  Facilitating  risk  assessments  and 

•  Preparing  risk  briefings,  reports,  and  documents  required  for  Program  Reviews 
and  the  acquisition  Milestone  decision  processes. 

2.3.2  Program  Integrating  Integrated  Product  Team  (PIIPT) 

The  PIIPT  is  responsible  for  complying  with  the  DoD  risk  management  policy  and  for 
structuring  an  efficient  and  useful  XYZ  risk  management  approach.  The  Program  Man¬ 
ager  is  the  Chair  of  the  PIIPT.  The  PIIPT  membership  may  be  adjusted  but  is  initially 
established  as  the  chairs  of  the  Program  IPTs,  designated  sub-tier  IPTs,  and  the  Heads  of 
PMO  Functional  Offices. 

2.3.3  PIPTs 

The  program  IPTs  are  responsible  for  implementing  risk  management  tasks  per  this  plan. 
This  includes  the  following  responsibilities: 

•  Review  and  recommend  to  the  Risk  Management  Coordinator  changes  on  the  over¬ 
all  risk  management  approach  based  on  lessons  learned. 

•  Quarterly,  or  as  directed,  update  the  program  risk  assessments  made  during 
Phase  I. 

•  Review  and  be  prepared  to  justify  the  risk  assessments  made  and  the  risk  mitiga¬ 
tion  plans  proposed. 

•  Report  risk  to  the  Program  Manager/Program  Director,  with  information  to  the 
Risk  Management  Coordinator  via  Risk  Information  Forms  (RIFs). 

•  Ensure  that  risk  is  a  consideration  at  each  Program  and  Design  Review. 

•  Ensure  Design/Build  Team  responsibilities  incorporate  appropriate  risk  manage¬ 
ment  tasks. 

2.3.4  XYZ  Independent  Risk  Assessors 

Independent  Assessors  made  a  significant  contribution  to  the  XYZ  Milestone  I  risk  assess¬ 
ments.  The  use  of  independent  assessments  as  a  means  of  ensuring  that  all  risk  areas  are 
identified  will  continue,  when  necessary. 

2.3.5  Other  Risk  Assessment  Responsibilities 

The  Risk  Assessment  responsibilities  of  other  Systems  Command  codes.  Service  Field 
Activities,  Design/ Build  Teams,  and  Contractors  will  be  as  described  in  Memoranda  of 
Agreement  (MOAs),  Memoranda  of  Understanding  (MOUs),  Systems  Command  Task¬ 
ing,  or  contracts.  This  RMP  should  be  used  as  a  guide  for  XYZ  risk  management  efforts. 
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2.3.6  User  Participation 

The  Requirements  Organization  (specific  code)  is  the  focal  point  for  providing  the  Pro¬ 
gram  Executive  Officer  or  the  Project  Manager  with  user  identified  risk  assessments. 

2.3.7  Risk  Training 

The  key  to  the  success  of  the  risk  efforts  is  the  degree  to  which  all  members  of  the  team, 
both  Government  and  contractor  are  properly  trained.  The  XYZ  Program  Office  will  pro¬ 
vide  risk  training,  or  assign  members  to  training  classes,  during  acquisition  Phases  I  and 
II.  Key  personnel  with  XYZ  management  or  assessment  responsibilities  are  required  to 
attend.  All  members  of  the  team  will  receive,  at  a  minimum,  basic  risk  management  train¬ 
ing.  XYZ  sponsored  training  is  planned  to  be  presented  according  to  the  schedule  pro¬ 
vided  in  Annex  X  (not  provided). 

3.0  RISK  MANAGEMENT  PROCESS  AND  PROCEDURES 
3.1  OVERVIEW 

This  section  describes  XYZ  program's  risk  management  process  and  provides  an  over¬ 
view  of  the  XYZ  risk  management  approach.  The  Defense  Acquisition  Deskbook  defines  risk 
management  as  "the  act  or  practice  of  controlling  risk.  It  includes  risk  planning,  assessing 
risk  areas,  developing  risk-handling  options,  monitoring  risks  to  determine  how  risks 
have  changed,  and  documenting  the  overall  risk  management  program."  Figure 
3-1  shows,  in  general  terms,  the  overall  risk  management  process  that  will  be  followed  in 
the  XYZ  program.  This  process  follows  DoD  and  Service  policies  and  guidelines  and  in¬ 
corporates  ideas  found  in  other  sources.  Each  of  the  risk  management  functions  shown  in 
Figure  3-1  is  discussed  in  the  following  paragraphs,  along  with  specific  procedures  for 
executing  them. 


•  Requirements  •  Risk  Events  •  Mitigation  Tasks  •  Metrics 

•  Responsibilities  •  Analysis  •  Metrics  -Track  Status 

•  Definitions  •  Update  Assessments  •  Report  •  Report 

•  Resources  *  Document  Findings 

•  Procedures 


Figure  3-1 .  Risk  Management  Structure 
(also  referred  to  as  The  Risk  Management  Process  Model) 
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3.2  RISK  PLANNING 

3.2.1  Process 

Risk  planning  consists  of  the  up-front  activities  necessary  to  execute  a  successful  risk 
management  program.  It  is  an  integral  part  of  normal  program  planning  and  manage¬ 
ment.  The  planning  should  address  each  of  the  other  risk  management  functions,  result¬ 
ing  in  an  organized  and  thorough  approach  to  assess,  handle,  and  monitor  risks.  It  should 
also  assign  responsibilities  for  specific  risk  management  actions  and  establish  risk  report¬ 
ing  and  documentation  requirements.  This  RMP  serves  as  the  basis  for  all  detailed  risk 
planning,  which  must  be  continuous. 

3.2.2  Procedures 

3.2.2.1  Responsibilities.  Each  IPT  is  responsible  for  conducting  risk  planning,  using 
this  RMP  as  the  basis.  The  planning  will  cover  all  aspects  of  risk  management  to  include 
assessment,  handling  options,  and  monitoring  of  risk  mitigation  activities.  The  Program 
Risk  Management  Coordinator  will  monitor  the  planning  activities  of  the  IPTs  to  ensure 
that  they  are  consistent  with  this  RMP  and  that  appropriate  revisions  to  this  plan  are 
made  when  required  to  reflect  significant  changes  resulting  from  the  IPT  planning  efforts. 

Each  person  involved  in  the  design,  production,  operation,  support,  and  eventual  dis¬ 
posal  of  the  XYZ  system  or  any  of  its  systems  or  components  is  a  part  of  the  risk  manage¬ 
ment  process.  This  involvement  is  continuous  and  should  be  considered  a  part  of  the 
normal  management  process. 

3.2.2.2  Resources  and  Training.  An  effective  risk  management  program  requires  resources. 
As  part  of  its  planning  process,  each  IPT  will  identify  the  resources  required  to  implement 
the  risk  management  actions.  These  resources  include  time,  material,  personnel,  and  cost. 
Training  is  major  consideration.  All  IPT  members  should  receive  instruction  on  the  funda¬ 
mentals  of  risk  management  and  special  training  in  their  area  of  responsibility,  if  necessary. 

3.2.2.3  Documentation  and  Reporting.  This  RMP  establishes  the  basic  documentation 
and  reporting  requirements  for  the  program.  IPTs  should  identify  any  additional  require¬ 
ments  that  might  be  needed  to  effectively  manage  risk  at  their  level.  Any  such  additional 
requirements  must  not  conflict  with  the  basic  requirements  in  this  RMP. 

3.2.2.4  Metrics.  Each  IPT  should  establish  metrics  that  will  measure  the  effectiveness  of 
their  planned  risk-handling  options.  See  Annex  C  for  an  example  of  metrics  that  may  be 
used. 

3.2.2.5  Risk  Planning  Tools.  The  following  tools  can  be  useful  in  risk  planning.  It  may  be 
useful  to  provide  this  information  to  the  contractors  to  help  them  understand  the  XYZ 
program's  approach  to  managing  risk.  This  list  is  not  meant  to  be  exclusive. 

•  DoD  Manual  4245.7-M,  a  DoD  guide  for  assessing  process  technical  risk. 

•  The  Navy's  Best  Practices  Manual,  NAVSO  P-6071,  provides  additional  insight  into 
each  of  the  Templates  in  DoD  4245.7-M  and  a  checklist  for  each  template. 
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•  Program  Manager's  Work  Station  (PMWS)  software,  may  be  useful  to  some  risk 
assessors.  PMWS  has  a  Risk  Assessment  module  based  on  the  Template  Manual  and  Best 
Practices  Manual. 

•  Commercial  and  Government  developed  risk  management  software. 

The  latter  includes  Government  software,  such  as  Risk  Matrix  developed  by  Mitre  Corpo¬ 
ration  for  the  Air  Force  and  the  New  Attack  Submarine's  On-Line  Risk  Data  Base  (OLRDB). 

3.2.2.6  Plan  Update.  This  RMP  will  be  updated,  if  necessary,  on  the  following  occasions: 
(1)  whenever  the  acquisition  strategy  changes,  or  there  is  a  major  change  in  program  em¬ 
phasis;  (2)  in  preparation  for  major  decision  points;  (3)  in  preparation  for  and  immedi¬ 
ately  following  technical  audits  and  reviews;  (4)  concurrent  with  the  review  and  update 
of  other  program  plans;  and  (5)  in  preparation  for  a  POM  submission. 

3.3  RISK  ASSESSMENT 

The  risk  assessment  process  includes  the  identification  of  critical  risk  events /processes, 
which  could  have  an  adverse  impact  on  the  program,  and  the  analyses  of  these  events/ 
processes  to  determine  the  likelihood  of  occurrence/ process  variance  and  consequences. 
It  is  the  most  demanding  and  time-consuming  activity  in  the  risk  management  process. 

3.3.1  Process 

3.3.1.1  Identification.  Risk  identification  is  the  first  step  in  the  assessment  process.  The 
basic  process  involves  searching  through  the  entire  XYZ  program  to  determine  those  criti¬ 
cal  events  that  would  prevent  the  program  from  achieving  its  objectives.  All  identified 
risks  will  be  documented  in  the  RMIS,  with  a  statement  of  the  risk  and  a  description  of  the 
conditions  or  situations  causing  concern  and  the  context  of  the  risk. 

Risks  will  be  identified  by  all  IPTs  and  by  any  individual  in  the  program.  The  lower-level 
IPTs  can  identify  significant  concerns  earlier  than  otherwise  might  be  the  case  and  iden¬ 
tify  those  events  in  critical  areas  that  must  be  dealt  with  to  avoid  adverse  consequences. 
Likewise,  individuals  involved  in  the  detailed  and  day-to-day  technical,  cost,  and  sched¬ 
uling  aspects  of  the  program  are  most  aware  of  the  potential  problems  (risks)  that  need  to 
be  managed. 

3.3.1.2  Analysis.  This  process  involves: 

•  Identification  of  WBS  elements 

•  Evaluation  of  the  WBS  elements  using  the  risk  areas  to  determine  risk  events 

•  Assignment  of  likelihood  /probability  and  consequence  to  each  risk  event  to  estab¬ 
lish  a  risk  rating 

•  Prioritization  of  each  risk  event  relative  to  other  risks. 

Risk  analysis  should  be  supported  by  a  study,  test  results,  modeling  and  simulation,  trade 
study,  the  opinion  of  a  qualified  expert  (to  include  justification  of  his  or  her  judgment),  or 
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any  other  accepted  analysis  technique.  The  DoD  Acquisition  Deskbook,  Section  2524.2  de¬ 
scribes  a  number  of  analysis  techniques  that  may  be  useful.  Evaluators  should  identify  all 
assumptions  made  in  assessing  risk.  When  appropriate,  a  sensitivity  analysis  should  be 
done  on  assumptions. 

Systems  engineering  analysis,  risk  assessments,  and  manpower  risk  assessments  provide 
additional  information  that  must  be  considered.  This  includes,  among  other  things,  envi¬ 
ronmental  impact,  system  safety  and  health  analysis,  and  security  considerations.  Classi¬ 
fied  programs  may  experience  difficulties  in  access,  facilities,  and  visitor  control  that  can 
introduce  risk  and  must  be  considered. 

The  analysis  of  individual  risk  will  be  the  responsibility  of  the  IPT  identifying  the  risk,  or 
the  IPT  to  which  the  risk  has  been  assigned.  They  may  use  external  resources  for  assis¬ 
tance,  such  as  field  activities.  Service  laboratories,  and  contractors.  The  results  of  the  analysis 
of  all  identified  risks  must  be  documented  in  the  RMIS. 

3.3.2  Procedures 

3.3.2.1  Assessments— General.  Risk  assessment  is  an  iterative  process,  with  each  assess¬ 
ment  building  on  the  results  of  previous  assessments.  The  current  baseline  assessment  is  a 
combination  of  the  risk  assessment  delivered  by  the  contractors  as  part  of  Phase  0,  the 
program  office  process  risk  assessment  done  before  Milestone  I,  and  the  post-award  Inte¬ 
grated  Baseline  Review  (IBR). 

For  the  program  office,  unless  otherwise  directed  in  individual  tasking,  program  level 
risk  assessments  will  be  presented  at  each  Program  Review  meeting  with  a  final  update 
not  later  than  6  months  before  the  next  scheduled  Milestone  decision.  The  primary  source 
of  information  for  the  next  assessment  will  be  the  current  assessment  baseline,  and  exist¬ 
ing  documentation  such  as.  Phase  0  study  results,  the  design  mission  profile  that  was 
done  as  part  of  Phase  0,  the  IBR,  which  will  be  conducted  immediately  after  Phase  I  con¬ 
tract  award,  the  contract  WBS  that  is  part  of  the  IBR,  industry  best  practices  as  described 
in  the  PMWS  Knowledge  base,  the  ORD,  the  Acquisition  Program  Baseline  (APB),  and 
any  contractor  design  documents. 

IPTs  should  continually  assess  the  risks  in  their  areas,  reviewing  risk-mitigation  actions 
and  the  critical  risk  areas  whenever  necessary  to  assess  progress.  For  contractors,  risk 
assessment  updates  should  be  made  as  necessary. 

The  risk  assessment  process  is  intended  to  be  flexible  enough  so  that  field  activities,  ser¬ 
vice  laboratories,  and  contractors  may  use  their  judgment  in  structuring  procedures  con¬ 
sidered  most  successful  in  identifying  and  analyzing  all  risk  areas.  , 

3.3.2.2  Identification.  Following  is  a  description  of  step-by-step  procedures  that  evalua¬ 
tors  may  use  as  a  guide  to  identify  program  risks. 

•  Step  One — Understand  the  requirements  and  the  program  performance  goals, 
which  are  defined  as  thresholds  and  objectives  (see  5000.2-R).  Describe  the  operational 
(functional  and  environmental)  conditions  under  which  the  values  must  be  achieved  by 
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referring  or  relating  to  design  documents.  The  ORD  and  APB  contain  Key  Performance 
Parameters  (KPPs). 

•  Step  Two — Determine  the  engineering  and  manufacturing  processes  that  are  needed 
to  design,  develop,  produce,  and  support  the  system.  Obtain  industry  best  practices  for 
these  processes. 

•  Step  Three — Identify  contract  WBS  elements  (to  include  products  and  processes). 

•  Step  Four — Evaluate  each  WBS  element  against  sources/areas  of  risk  described  in 
Table  4-2  of  the  DSMC  Risk  Management  Guide,  plus  other  sources/areas  as  appropriate. 

•  Step  Five — Assign  a  probability  and  consequence  to  each  risk  event 

•  Step  Six— Prioritize  the  risk  events. 

Following  are  indicators  that  IPTs  may  find  helpful  in  identifying  and  assessing  risk: 

•  Lack  of  Stability,  Clarity,  or  Understanding  of  Requirements:  Requirements  drive 
the  design  of  the  system.  Changing  or  poorly  stated  requirements  guarantees  the  intro¬ 
duction  of  performance,  cost,  and  schedule  problems. 

•  Failure  to  Use  Best  Practices  virtually  assures  that  the  program  will  experience 
some  risk.  The  further  a  contractor  deviates  from  best  practices,  the  higher  the  risk. 

•  New  Processes  should  always  be  suspect,  whether  they  are  related  to  design,  analy¬ 
sis,  or  production.  Until  they  are  validated,  and  until  the  people  who  implement  them 
have  been  trained  and  have  experience  in  successfully  using  the  process,  there  is  risk. 

•  Any  Process  Lacking  Rigor  should  also  be  suspect;  it  is  inherently  risky.  To  have 
rigor,  a  process  should  be  mature  and  documented,  it  should  have  been  validated,  and  it 
should  be  strictly  followed. 

•  Insufficient  Resources:  People,  funds,  schedule,  and  tools  are  necessary  ingredi¬ 
ents  for  successfully  implementing  a  process.  If  any  are  inadequate,  to  include  the  qualifi¬ 
cations  of  the  people,  there  is  risk. 

•  Test  Failure  may  indicate  corrective  action  is  necessary.  Some  corrective  actions 
may  not  fit  available  resources,  or  the  schedule,  and  (for  other  reasons  as  well)  may  con¬ 
tain  risk. 

•  Qualified  Supplier  Availability:  A  supplier  not  experienced  with  the  processes  for 
designing  and  producing  a  specific  product  is  not  a  qualified  supplier  and  is  a  source  of 
risk. 

•  Negative  Trends  or  Forecasts  are  cause  for  concern  (risk)  and  may  require  specific 
actions  to  turn  around. 

There  are  a  number  of  techniques  and  tools  available  for  identifying  risks.  Among  them 
are: 
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•  Best  Judgment:  The  knowledge  and  experience  of  the  collective,  multi-disciplined 
Integrated  Project  Team  (IPT)  members  and  the  opinion  of  subject-matter  experts  (SMEs) 
are  the  most  common  source  of  risk  identification. 

•  Lessons  Learned  from  similar  processes  can  serve  as  a  baseline  for  the  successful 
way  to  achieve  requirements.  If  there  is  a  departure  from  the  successful  way,  there  may  be 
risk. 

•  DoD  4245.7-M,  "Transition  from  Development  to  Production,"  is  often  called  the 

"Templates"  book  because  it  identifies  technical  risk  areas  and  provides,  in  "bullet"  form, 
suggestions  for  avoiding  those  risks.  It  focuses  on  the  technical  details  of  product  design, 
test,  and  production  to  help  managers  proactively  manage  risk.  It  also  includes  chapters 
on  Facilities,  Logistics,  and  Management,  which  make  this  a  useful  tool  in  identifying 
weak  areas  of  XYZ  planned  processes  early  enough  to  implement  actions  needed  to  avoid 
adverse  consequences. 

•  The  NAVSO  P-6071  Best  Practices  Manual  was  developed  by  the  Navy  to  add 
depth  to  the  Template  Book,  DoD  4245.7-M. 

•  Critical  Program  Attributes  are  metrics  that  the  program  office  developed  to  mea¬ 
sure  progress  toward  meeting  our  objectives.  Team  members,  IPTs,  functional  managers, 
contractors,  etc.,  may  develop  their  own  metrics  to  support  these  measurements.  The  at¬ 
tributes  may  be  specification  requirements,  contract  requirements,  or  measurable  param¬ 
eters  from  any  agreement  or  tasking.  The  idea  is  to  provide  a  means  to  measure  whether 
we  are  on  track  in  achieving  our  objectives. 

•  Methods  and  Metrics  for  Product  Success  is  a  manual  published  by  the  Office  of 
the  Assistant  Secretary  of  the  Navy  (RDA)  Product  Integrity  Directorate.  It  highlights  ar¬ 
eas  related  to  design,  test,  and  production  processes  where  problems  are  most  often  found 
and  metrics  for  the  measurement  of  effectiveness  of  the  processes.  It  also  describes  the 
software  tool.  Program  Manager's  Work  Station  (PMWS).  See  next  paragraph. 

•  PMWS  contains  risk  management  software,  "Technical  Risk  Identification  and  Miti¬ 
gation  System  (TRIMS)  and  Knowledgebase."  They  provide  a  tailorable  management  sys¬ 
tem  based  on  NAVSO  P-6071  and  DoD  4245.7-M.  The  PMWS  provides  a  compact  disk 
(CD)  that  contains  the  necessary  programs  for  assessing  a  program's  risk  and  software  for 
program  management.  PMWS  can  be  obtained  by  calling  the  Best  Manufacturing  Pro¬ 
gram  (BMP)  Office  at  (301)  403-8100. 

•  New  Nuclear  Submarine  (NSSN)  On-Line  Risk  Database  (ONLRB)  is  a  software 
tool  may  be  used  to  support  the  XYZ  Risk  Management  Process.  The  tool  helps  IPTs  in  the 
identification  and  assessment  of  risk  and  management  of  handling  efforts. 

•  Risk  Matrix  is  another  candidate  for  use  by  the  PMO.  It  is  an  automated  tool, 
developed  by  Mitre  Corporation,  that  supports  a  structured  approach  for  identifying 
risk  and  assessing  its  potential  program  impact.  It  is  especially  helpful  for  prioritizing 
risks. 
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•  Requirements  Documents  describe  the  output  of  our  efforts.  IPT  efforts  need  to  be 
monitored  continuously  to  ensure  requirements  are  met  on  time  and  within  budget.  When 
they  aren't,  there  is  risk. 

•  Contracting  for  Risk  Management  helps  ensure  the  people  involved  with  the  de¬ 
tails  of  the  technical  processes  of  design,  test,  and  production  are  involved  with  managing 
risk.  The  principle  here  is  that  those  performing  the  technical  details  are  normally  the  first 
ones  to  know  when  risks  exist. 

•  Quality  Standards,  such  as  IS09000,  ANSI/ASQC  Q  9000,  MIL-HDBK  9000,  and 

others  describe  processes  for  developing  and  producing  quality  products.  Comparing  our 
processes  with  these  standards  can  highlight  areas  we  may  want  to  change  to  avoid  risk. 

•  Use  of  Independent  Risk  Assessors  is  a  method  to  help  ensure  all  risk  is  identified. 
The  knowledgeable,  experienced  people  are  independent  from  the  management  and  ex¬ 
ecution  of  the  processes  and  procedures  being  reviewed.  Independent  assessment  pro¬ 
motes  questions  and  observations  not  otherwise  achievable. 

3.3.2.3  Analysis.  Risk  analysis  is  an  evaluation  of  the  identified  risk  events  to  determine 
possible  outcomes,  critical  process  variance  from  known  best  practices,  the  likelihood  of 
those  events  occurring,  and  the  consequences  of  the  outcomes.  Once  this  information  has 
been  determined,  the  risk  event  may  be  rated  against  the  program's  criteria  and  an  overall 
assessment  of  low,  moderate,  or  high  assigned.  Figure  3-2  depicts  the  risk  analysis  process 
and  procedures. 

Critical  Process  Variance.  For  each  process  risk  related  event  identified,  the  variance  of 
the  process  from  known  standards  or  best  practices  must  be  determined.  As  shown  in 
Figure  3-2,  there  are  five  levels  (a-e)  in  the  XYZ  risk  assessment  process,  with  the  corre¬ 
sponding  criteria  of  Minimal,  Small,  Acceptable,  Large,  and  Significant.  If  there  is  no  variance 
then  there  is  no  risk. 

Likelihood/Probability.  For  each  risk  area  identified,  the  likelihood  the  risk  will  happen 
must  be  determined.  As  shown  in  Figure  3-2,  there  are  five  levels  (a-e)  in  the  XYZ  risk 
assessment  process,  with  the  corresponding  subjective  criteria  of  Remote,  Unlikely,  Likely, 
Highly  Likely,  and  Near  Certainty.  If  there  is  zero  likelihood  of  an  event,  there  is  no  risk  per 
our  definition. 

Consequence.  For  each  risk  area  identified,  the  following  question  must  be  answered: 
Given  the  event  occurs,  what  is  the  magnitude  of  the  consequence?  As  shown  in  the  figure,  there 
are  five  levels  of  consequence  (a-e).  "Consequence"  is  a  multifaceted  issue.  For  this  program, 
there  are  four  areas  that  we  will  evaluate  when  determining  consequence:  technical  per¬ 
formance,  schedule,  cost,  and  impact  on  other  teams.  At  least  one  of  the  four  consequence 
areas  needs  to  apply  for  there  to  be  risk;  if  there  is  no  adverse  consequence  in  any  of  the 
areas,  there  is  no  risk. 

•  Technical  Performance:  This  category  includes  all  requirements  that  are  not  in¬ 
cluded  in  the  other  three  metrics  of  the  Consequence  table.  The  wording  of  each  level  is 
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Level 

What  is  the  Likelihood  the 
Risk  Event  Will  Happen? 
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'  Process  Variance  refers  to 
|  deviation  from  best  practices. 

|  Likelihood/Probability  refers  to 
.  risk  events. 


Consequence 


RISK  ASSESSMENT 

R  HIGH— Unacceptable,  Major 
disruption  likely.  Different 
approach  required.  Priority 
management  attention 
required. 

Y  MODERATE— Some 
disruption.  Different 
approach  may  be  required. 
Additional  management 
attention  may  be  needed. 

G  LOW— Minimum  impact. 
Minimum  oversight  needed 
to  ensure  risk  remains  low. 


Level 

Technical  and  / 

Performance  or 

and  / 

Schedule  or 

and  / 

Cost  or 

Impact  on 
Other  Teams 

.1 

Minimal  or  no  impact 

Minimal  or  no  impact 

Minimal  or  no  impact 

None 

2 

Acceptable  with  some 
reduction  in  margin 
need  dates 

Additional  resources 
required;  able  to  meet 

<5% 

Some  impact 

3 

Acceptable  with  significant 
reduction  in  margin 

Minor  slip  in  key  milestones; 
not  able  to  meet  need  date 

5-7% 

Moderate  impact 

4 

Acceptable;  no  remaining 
margin 

Major  slip  in  key  milestone 
or  critical  path  impacted 

7-10% 

Major  impact 

5 

Unacceptable 

major  program  milestone 

Can’t  achieve  key  team  or 

>10% 

Unacceptable 

Figure  3-2.  Risk  Assessment  Process 

oriented  toward  design  processes,  production  processes,  life  cycle  support,  and  to 
retirement  of  the  system.  For  example,  the  word  "margin"  could  apply  to  weight  margin 
during  design,  safety  margin  during  testing,  or  machine  performance  margin  during  pro¬ 
duction. 


•  Schedule:  The  words  used  in  the  Schedule  column,  as  in  all  columns  of  the  Conse¬ 
quence  table,  are  meant  to  be  universally  applied.  Avoid  excluding  a  consequence  level 
from  consideration  just  because  it  doesn't  match  your  team's  specific  definitions.  In  other 
words,  phrases  such  as  need  dates,  key  milestones,  critical  path,  and  key  team  milestones 
are  meant  to  apply  to  all  IPTs. 

•  Cost:  Since  costs  vary  from  component  to  component  and  process  to  process,  the 
percentage  criteria  shown  in  the  figure  may  not  strictly  apply  at  the  lower  levels  of  the 
WBS.  These  team  leaders  can  set  the  percentage  criteria  that  best  reflects  their  situation. 
However,  when  costs  are  rolled  up  at  higher  levels  (e.g.,  Program),  the  following  defini¬ 
tions  will  be  used:  Level  1— no  change,  Level  2— <5%,  Level  3-5-7%,  Level  4-7-10%, 
and  Level  5 — >10%. 


•  Impact  on  Other  Teams:  Both  the  consequence  of  a  risk  and  the  mitigation  actions 
associated  with  reducing  the  risk  may  impact  another  team.  This  may  involve  additional 
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coordination  or  management  attention  (resources)  and  may  therefore  increase  the  level  of 
risk.  This  is  especially  true  of  common  technical  processes. 

Risk  Rating.  Probability  and  consequence  should  not  always  be  considered  equally;  for 
example,  there  may  be  consequences  so  severe  that  it  is  considered  high  risk  even  though 
the  probability  to  achieve  a  particular  outcome  is  low.  After  deciding  a  level  of  process 
variance/ likelihood  (a  through  e)  and  a  level  of  consequence  (a  through  e),  enter  the  As¬ 
sessment  Guide  portion  of  Figure  3-2  to  obtain  a  risk  rating  (green  =  LOW,  yellow  =  MOD, 
and  red  =  HIGH).  For  example;  consequence /process  variartee/likelihood  level  2b  corre¬ 
sponds  to  LOW  risk,  level  3d  corresponds  to  MOD  risk,  level  5c  corresponds  to  HIGH 
risk.  After  obtaining  the  risk  rating,  make  a  subjective  comparison  of  the  risk  event  with 
the  applicable  rating  definition  in  Figure  3-2  (e.g.,  High=unacceptable,  major  disruptions, 
etc.).  There  should  be  a  close  match.  If  there  isn't,  consider  reevaluating  the  level  of  likeli¬ 
hood  or  consequence.  Those  risk  events  that  are  assessed  as  moderate  or  high  should  be 
submitted  to  the  XYZ  Risk  Management  Coordinator  on  a  RIF. 

Figure  3-2  is  useful  to  convey  information  to  decision  makers  and  will  be  used  primarily  for 
that  purpose.  The  PMO  will  use  the  Risk  Tracking  Report  and  Watch  List.  (See  Annex  D.) 

3.4  RISK  HANDLING 

3.4.1  Process 

After  the  program's  risks  have  been  identified  and  assessed,  the  approach  to  handling 
each  significant  risk  must  be  developed.  There  are  essentially  four  techniques  or  options 
for  handling  risks:  avoidance,  control,  transfer,  and  assumption.  For  all  identified  risks, 
the  various  handling  techniques  should  be  evaluated  in  terms  of  feasibility,  expected  ef¬ 
fectiveness,  cost  and  schedule  implications,  and  the  effect  on  the  system's  technical  per¬ 
formance,  and  the  most  suitable  technique  selected.  Section  2524.3  of  the  DoD  Acquisition 
Deskbook  contains  information  on  the  risk-handling  techniques  and  various  actions  that 
can  be  used  to  implement  them.  The  results  of  the  evaluation  and  selection  will  be  in¬ 
cluded  and  documented  in  the  RMIS  using  the  RIF.  This  documentation  will  include:  what 
must  be  done,  the  level  of  effort  and  materials  required,  the  estimated  cost  to  implement 
the  plan,  a  proposed  schedule  showing  the  proposed  start  date,  the  time  phasing  of  sig¬ 
nificant  risk  reduction  activities,  the  completion  date,  and  their  relationship  to  significant 
Program  activities /milestones  (an  example  is  provided  in  Annex  B),  recommended  metrics 
for  tracking  the  action,  a  list  of  all  assumptions,  and  the  person  responsible  for  imple¬ 
menting  and  tracking  the  selected  option. 

3.4.2  Procedures 

The  IPT  that  assessed  the  risk  is  responsible  for  evaluating  and  recommending  to  the  PM 
the  risk-handling  options  that  are  best  fitted  to  the  program's  circumstances.  Once  ap¬ 
proved,  these  are  included  in  the  program's  acquisition  strategy  or  management  plans,  as 
appropriate. 

For  each  selected  handling  option,  the  responsible  IPT  will  develop  specific  tasks  that,  when 
implemented,  will  handle  the  risk.  The  task  descriptions  should  explain  what  has  to  be  done. 
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the  level  of  effort,  and  identify  necessary  resources.  It  should  also  provide  a  proposed  sched¬ 
ule  to  accomplish  the  actions  including  the  start  date,  the  time  phasing  of  significant  risk 
reduction  activities,  the  completion  date,  and  their  relationship  to  significant  Program  activi¬ 
ties/milestones  (an  example  is  provided  in  Annex  B),  and  a  cost  estimate.  The  description  of 
the  handling  options  should  list  all  assumptions  used  in  the  development  of  the  handling 
tasks.  Assumptions  should  be  included  in  the  RIF.  Recommended  actions  that  require  resources 
outside  the  scope  of  a  contract  or  official  tasking  should  be  clearly  identified,  and  the  IPTs,  the 
risk  area,  or  other  handling  plans  that  may  be  impacted  should  be  listed. 

Reducing  requirements  as  a  risk  avoidance  technique  will  be  used  only  as  a  last  resort, 
and  then  only  with  the  participation  and  approval  of  the  user  s  representative. 

DoD  4245.7-M  Templates  and  NAVSO  P-6071  Best  Practices,  are  useful  in  developing  risk¬ 
handling  actions  for  design,  test,  or  manufacturing  process  risks. 

3.5  RISK  MONITORING 

3.5.1  Process 

Risk  monitoring  systematically  tracks  and  evaluates  the  performance  of  risk-handling 
actions.  It  is  part  of  the  PMO  function  and  responsibility  and  will  not  become  a  separate 
discipline.  Essentially,  it  compares  predicted  results  of  planned  actions  with  the  results 
actually  achieved  to  determine  status  and  the  need  for  any  change  in  risk-handling  ac¬ 
tions.  The  effectiveness  of  the  risk-monitoring  process  depends  on  the  establishment  of  a 
management  indicator  system  (metrics)  that  provides  accurate,  timely,  and  relevant  risk 
information  in  a  clear,  easily  understood  manner.  (See  Annex  D.)  The  metrics  selected  to 
monitor  program  status  must  adequately  portray  the  true  state  of  the  risk  events  and 
handling  actions.  Otherwise,  indicators  of  risks  that  are  about  to  become  problems  will  go 
undetected. 

To  ensure  that  significant  risks  are  effectively  monitored,  risk-handling  actions  (which  in¬ 
clude  specific  events,  schedules,  and  "success"  criteria)  will  be  reflected  in  integrated  pro¬ 
gram  planning  and  scheduling.  Identifying  these  risk  handling  actions  and  events  m  the 
context  of  Work  Breakdown  Structure  (WBS)  elements  establishes  a  linkage  between  them 
and  specific  work  packages,  making  it  easier  to  determine  the  impact  of  actions  on  cost, 
schedule,  and  performance.  The  detailed  information  on  risk-handling  actions  and  events 
will  be  included  in  the  RIF  for  each  identified  risk,  and  thus  be  resident  in  the  RMIS. 

3.5.2  Procedures 

The  functioning  of  IPTs  is  crucial  to  effective  risk  monitoring.  They  are  the  "front  line"  for 
obtaining  indications  that  risk-handling  efforts  are  achieving  their  desired  effects.  Each 
IPT  is  responsible  for  monitoring  and  reporting  the  effectiveness  of  the  handling  actions 
for  the  risks  assigned.  Overall  XYZ  program  risk  assessment  reports  will  be  prepared  by 
the  XYZ  Risk  Management  Coordinator  working  with  the  cognizant  IPT. 

Many  techniques  and  tools  are  available  for  monitoring  the  effectiveness  of  risk-han¬ 
dling  actions,  and  IPTs  must  ensure  that  they  select  those  that  best  suit  their  needs.  No 
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single  technique  or  tool  is  capable  of  providing  a  complete  answer — a  combination  must 
be  used.  At  a  minimum,  each  IPT  will  maintain  a  watch  list  of  identified  high  priority 
risks.  See  Section  2524.4  of  the  DoD  Acquisition  Deskbook  for  information  on  specific 
techniques. 

Risks  rated  as  Moderate  or  High  risk  will  be  reported  to  the  XYZ  Risk  Management  Coor¬ 
dinator,  who  will  also  track  them,  using  information  provided  by  the  appropriate  IPT, 
until  the  risk  is  considered  Low  and  recommended  for  "Close  Out."  The  IPT  that  initially 
reported  the  risk  retains  ownership  and  cognizance  for  reporting  status  and  keeping  the 
database  current.  Ownership  means  implementing  handling  plans  and  providing  peri¬ 
odic  status  of  the  risk  and  of  the  handling  plans.  Risk  will  be  made  an  agenda  item  at  each 
management  or  design  review,  providing  an  opportunity  for  all  concerned  to  offer  sug¬ 
gestions  for  the  best  approach  to  managing  risk.  Communicating  risk  increases  the 
program's  credibility  and  allows  early  actions  to  minimize  adverse  consequences. 

The  risk  management  process  is  continuous.  Information  obtained  from  the  monitoring 
process  is  fed  back  for  reassessment  and  evaluations  of  handling  actions.  When  a  risk  area 
is  changed  to  Low,  it  is  put  into  a  "Historical  File"  by  the  Risk  Management  Coordinator 
and  it  is  no  longer  tracked  by  the  XYZ  PMO.  The  "owners"  of  all  Low  risk  areas  will 
continue  monitoring  Low  risks  to  ensure  they  stay  Low. 

The  status  of  the  risks  and  the  effectiveness  of  the  risk-handling  actions  will  be  reported  to 
the  Risk  Management  Coordinator: 

•  Quarterly 

•  When  the  IPT  determines  that  the  status  of  the  risk  area  has  changed  significantly 
(as  a  minimum  when  the  risk  changes  from  high  to  moderate  to  low,  or  vice  versa) 

•  When  requested  by  the  Program  Manager. 

4.0  RISK  MANAGEMENT  INFORMATION  SYSTEM  AND  DOCUMENTATION 

The  XYZ  program  will  use  the  XXX  database  management  system  as  its  RMIS.  The  system 
will  contain  all  of  the  information  necessary  to  satisfy  the  program  documentation  and 
reporting  requirements. 

4.1  RISK  MANAGEMENT  INFORMATION  SYSTEM  (RMIS) 

The  RMIS  stores  and  allows  retrieval  of  risk-related  data.  It  provides  data  for  creating 
reports  and  serves  as  the  repository  for  all  current  and  historical  information  related  to 
risk.  This  information  will  include  risk  assessment  documents,  contract  deliverables,  if 
appropriate,  and  any  other  risk-related  reports.  The  PMO  will  use  data  from  the  RMIS  to 
create  reports  for  senior  management  and  retrieve  data  for  day-to-day  management  of 
the  program.  The  program  produces  a  set  of  standard  reports  for  periodic  reporting  and 
has  the  ability  to  create  ad  hoc  reports  in  response  to  special  queries.  See  Annex  D  for  a 
detailed  discussion  of  the  RMIS. 
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Data  are  entered  into  the  RMIS  using  the  Risk  Information  Form  (RIF).  The  RIF  gives 
members  of  the  project  team,  both  Government  and  contractors,  a  standard  format  for 
reporting  risk-related  information.  The  RIF  should  be  used  when  a  potential  risk  event  is 
identified  and  will  be  updated  as  information  becomes  available  as  the  assessment,  han¬ 
dling,  and  monitoring  functions  are  executed. 

4.2  RISK  DOCUMENTATION 

All  program  risk  management  information  will  be  documented,  using  the  RIF  as  the  stan¬ 
dard  RMIS  data  entry  form.  The  following  paragraphs  provide  guidance  on  documenta¬ 
tion  requirements  for  the  various  risk  management  functions. 

4.2.1  Risk  Assessment  Documentation 

Risk  assessments  form  the  basis  for  many  program  decisions.  From  time  to  time,  the  PM 
will  need  a  detailed  report  of  any  assessment  of  a  risk  event.  It  is  critical  that  all  aspects  of 
the  risk  management  process  are  documented. 

4.2.2  Risk  Handling  Documentation 

Risk-handling  documentation  will  be  used  to  provide  the  PM  with  the  information  he 
needs  to  choose  the  preferred  mitigation  option. 

4.2.3  Risk  Monitoring  Documentation 

The  PM  needs  a  summary  document  that  tracks  the  status  of  high  and  moderate  risks.  The 
Risk  Management  Coordinator  will  produce  a  risk  tracking  list,  for  example,  that  uses 
information  that  has  been  entered  from  the  RMIS.  This  document  will  be  produced  on  a 
monthly  basis. 

4.3  REPORTS 

Reports  are  used  to  convey  information  to  decision  makers  and  team  members  on  the 
status  of  the  program  and  the  effectiveness  of  the  risk  management  program.  Every  effort 
will  be  made  to  generate  reports  using  the  data  resident  in  the  RMIS. 

4.3.1  Standard  Reports 

The  RMIS  will  have  a  set  of  standard  reports.  If  IPTs  or  functional  managers  need  addi¬ 
tional  reports,  they  should  work  with  the  Risk  Management  Coordinator  to  create  them. 
Access  to  the  reporting  system  will  be  controlled;  however,  any  member  of  the  Govern¬ 
ment  or  contractor  team  may  obtain  a  password  to  gain  access  to  the  information.  See 
Annex  B  for  a  description  of  the  XYZ  program  reports. 

4.3.2  Ad  Hoc  Reports 

In  addition  to  standard  reports,  the  PMO  will  need  to  create  ad  hoc  reports  in  response 
to  special  queries.  The  Risk  Management  Coordinator  will  be  responsible  for  these 

reports. 
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ANNEX  A 

TO  XYZ  RISK  MANAGEMENT  PLAN 
—CRITICAL  PROGRAM  ATTRIBUTES— 


Description 


Speed  _ 


Weight 


Endurance 


Crew  Size 


Survivability 


Maneuverability 


Size 


Receiver  Range 


Transmitter  Range 


Data  Link  Operations 


Recovery  Time 


Initial  Setu 


Identification  Time 


Accuracy  Location 


Probability  of  Accurate  ID 


Responsible  IPT  |  Remarks 


Operating  and  Support  Costs 


Etc. 


Requirements  Stable 


Test  Plan  Approved 


Engine  Bench  Test 


Accuracy  Verified  by  Test  Data 
and  Analysis 


Toolproofing  Completed 


Logistics  Support  Reviewed  by 
User 
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ANNEX  B 

TO  XYZ  RISK  MANAGEMENT  PLAN 
—PROGRAM  RISK  REDUCTION  SCHEDULE— 


Accomplished 


■  Planned 


R 

I 


■  Determine  and  flowdown  requirements,  evaluate  potential  hardware  and  software  solutions.  Gather  data 
,  on  NDI  capabilities,  limitations,  evaluate  alternatives  and  pick  lower  risk  solutions. 

I 

mm  Simulations  to  evaluate  subsystem  interactions,  timing  issues.  Simulations  to  evaluate  target  sets, 
environment  effects. 

Preliminary  design  and  trade  studies  to  work  issues  such  as  temperature  and  shock  environments. 
Develop  baseline  design.  Reassess  risk. 


|  Get  hardware  and  software  in  place  for  pre-EMD  simulations.  Consolidate  team  structure  and  supplier. 


Hardware-in-the-Loop  (HWIL)  and  performance  prediction  demo.  Supporting  analyses  and  design 
studies. 


I 


■  Initiate  detailed  trade  studies  and  identify  alternatives.  Validate  and  implement  trade  study 
,  decisions  with  customer  on  IPD  teams  for  lower  risk  options.  Reassess  risk. 

I 

I 

Extensive  simulations  &  HWIL  testing.  Developmental  test  program,  supporting 


MS  II 

V 

SRR 

V 


|  analyses,  reviews  and  decisions. 

■ 

- - 1  Systems  integration  testing  (supported  by  continued  simulations)  to 

_ _ I  verify  design.  TAAF  program  with  selected  subsystems.  Reassess  risk. 

_ 

|  |  Qualification  testing. 

I 

■ 

|  |  Operational  testing  &  simulations. 

I 

—►  Production. 


PSR 

V 


CDR 

V 


PRR 

V 


MS  II 

V 


PD&RR 


EMD 


CyI  I  1996  |  1997  1998  1999  2000  2001  |  2002  |  2003 


XYZ  Program  Risk  Reduction  Schedule  (Example) 
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ANNEX  C 

TO  XYZ  RISK  MANAGEMENT  PLAN 
—PROGRAM  METRIC  EXAMPLES— 

Examples  of  Product-Related  Metrics _ 


Enqineering 

Requirements 

Production 

Support 

Key  Design  Parameters 
Weight 

Size 

Endurance 

Range 

Design  Maturity 

Open  problems  reports 
Number  of  engineering 
change  proposals 
Number  of  drawings 
released 

Failure  activities 

Computer  Resource 
Utilization 

Etc. 

Requirements 

Traceability 

Requirements  Stability 
Threat  Stability 

Design  Mission  Profile 

Manufacturing  Yields 
Incoming  Material  Yields 
Delinquent  Requisitions 
Unit  Production  Cost 
Process  Proofing 

Waste 

Personnel  Stability 

Special  Tools  and  Test 
Equipment 

Support  Infrastructure 
Footprint 

Manpower  Estimates 

Examples  of  Process  Metrics 


Design 

Requirements 

Trade 

Studies 

Design 

Process 

integrated  Test 
Plan 

Failure 

Reporting 

System 

Manufacturing 

Plan 

Development 
of  requirements 
traceability  plan 
Development 
of  specification 
tree 

Specifications 
reviewed  for: 
Definition  of 
all  use 

environments 
Definition  of 
all  functional 
requirements 
for  each 
mission 
performed 

Users  needs 
prioritized 
Alternative 
system  con¬ 
figurations 
selected 

Test  methods 
selected 

Design  require¬ 
ments  stability 
Producibility 
analysis 
conducted 
Design  ana¬ 
lyzed  for: 

Cost 

Parts 

reduction 

Manufac¬ 

turability 

Testability 

All  develop¬ 
mental  tests  at 
system  and 
subsystem 
level  identified 
Identification  of 
who  will  do  test 
(Government, 
contractor, 
supplier) 

Contractor 
corporate-level 
management 
involved  in 
failure  reporting 
and  corrective 
action  process 
Responsibility 
for  analysis 
and  corrective 
action  assigned 
to  specific 
individual  with 
close-out  date 

Plan  docu¬ 
ments  methods 
by  which  de¬ 
sign  to  be  built 
Plan  contains 
sequence  and 
schedule  of 
events  at  con¬ 
tractor  and 
sub-contractor 
levels  that 
defines  use  of 
materials,  fab¬ 
rication  flow, 
test  equipment, 
tools,  facilities, 
and  personnel 
Reflects  manu¬ 
facturing  in¬ 
clusion  in  de¬ 
sign  process. 
Includes  identi¬ 
fication  and  as¬ 
sessment  of 
design  facilities 
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Examples  of  Cost  and  Schedule  Metrics 


Cost 

Schedule 

Cost  variance 

Cost  performance  index 
Estimate  at  completion 
Management  reserve 

Schedule  variance 

Schedule  performance  index 

Design  schedule  performance 
Manufacturing  schedule  performance 
Test  schedule  performance 
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ANNEX  D 

TO  XYZ  RISK  MANAGEMENT  PLAN 
—MANAGEMENT  INFORMATION  SYSTEM  AND  DOCUMENTATION— 


1.0  DESCRIPTION 

In  order  to  manage  risk,  we  need  a  database  management  system  that  stores  and  allows 
retrieval  of  risk-related  data.  The  Risk  Management  Information  System  provides  data 
for  creating  reports  and  serves  as  the  repository  for  all  current  and  historical  information 
related  to  risk.  This  information  may  include  risk  assessment  documents,  contract 
deliverables,  if  appropriate,  and  any  other  risk-related  reports.  The  Risk  Management 
Coordinator  is  responsible  for  the  overall  maintenance  of  the  RMIS,  and  he  or  his  desig¬ 
nee  are  the  only  persons  who  may  enter  data  into  the  database. 

The  RMIS  will  have  a  set  of  standard  reports.  If  IPTs  or  functional  managers  need  addi¬ 
tional  reports,  they  should  work  with  the  Risk  Management  Coordinator  to  create  them. 
Access  to  the  reporting  system  will  be  controlled;  however,  any  member  of  the  Govern¬ 
ment  or  contractor  team  may  obtain  a  password  to  gain  access  to  the  information. 

In  addition  to  standard  reports,  the  PMO  will  need  to  create  ad  hoc  reports  in  response  to 
special  queries  etc.  The  Risk  Management  Coordinator  will  be  responsible  for  these  re¬ 
ports.  Figure  3-3  shows  a  concept  for  a  management  and  reporting  system. 


Figure  3-3.  Conceptual  Risk  Management  and  Reporting  System 


2.0  RISK  MANAGEMENT  REPORTS— XYZ  PROGRAM 

The  following  are  examples  of  basic  reports  that  a  PMO  may  use  to  manage  its  risk  pro¬ 
gram.  Each  office  should  coordinate  with  the  Risk  Management  Coordinator  to  tailor  and 
amplify  them,  if  necessary,  to  meets  its  specific  needs. 
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2.1  RISK  INFORMATION  FORM 

The  PMO  needs  a  document  that  serves  the  dual  purpose  of  a  source  of  data  entry  infor¬ 
mation  and  a  report  of  basic  information  for  the  IPTs,  etc.  The  Risk  Information  Form  (RIF) 
serves  this  purpose.  It  gives  members  of  the  project  team,  both  Government  and  contrac¬ 
tors,  a  format  for  reporting  risk-related  information.  The  RIF  should  be  used  when  a  po¬ 
tential  risk  event  is  identified  and  updated  over  time  as  information  becomes  available 
and  the  status  changes.  As  a  source  of  data  entry,  the  RIF  allows  the  database  administra¬ 
tor  to  control  entries.  The  format  for  a  RIF  is  included  on  page  B-30. 

2.2  RISK  ASSESSMENT  REPORT 

Risk  assessments  form  the  basis  for  many  program  decisions,  and  the  PM  may  need  a 
detailed  report  of  assessments  of  a  risk  event  that  has  been  done.  A  Risk  Assessment  Re¬ 
port  (RAR)  is  prepared  by  the  team  that  assessed  a  risk  event  and  amplifies  the  informa¬ 
tion  in  the  RIF.  It  documents  the  identification,  analysis,  and  handling  processes  and  re¬ 
sults.  The  RAR  amplifies  the  summary  contained  in  the  RIF,  is  the  basis  for  developing 
risk-handling  plans,  and  serves  as  a  historical  recording  of  program  risk  assessment.  Since 
RARs  may  be  large  documents,  they  may  be  stored  as  files.  RARs  should  include  informa¬ 
tion  that  links  it  to  the  appropriate  RIF. 

2.3  RISK-HANDLING  DOCUMENTATION 

Risk-handling  documentation  may  be  used  to  provide  the  PM  with  information  he  needs 
to  choose  the  preferred  mitigation  option  and  is  the  basis  for  the  handling  plan  summary 
contained  in  the  RIF.  This  document  describes  the  examination  process  for  risk-handling 
options  and  gives  the  basis  for  the  selection  of  the  recommended  choice.  After  the  PM 
chooses  an  option,  the  rationale  for  that  choice  may  be  included.  There  should  be  a  time- 
phased  plan  for  each  risk-mitigation  task.  Risk-handling  plans  are  based  on  results  of  the 
risk  assessment.  This  document  should  include  information  that  links  it  to  the  appropri¬ 
ate  RIF. 

2.4  RISK  MONITORING  DOCUMENTATION 

The  PM  needs  a  summary  document  that  tracks  the  status  of  high  and  moderate  risks.  The 
XYZ  program  will  use  a  risk-tracking  list  that  contains  information  that  has  been  entered 
from  the  RIF.  An  example  of  the  tracking  report/list  is  shown  on  page  B-31. 

3.0  DATABASE  MANAGEMENT  SYSTEM  (DBMS) 

The  XYZ  Risk  Management  Information  System  (RMIS)  provides  the  means  to  enter  and 
access  data,  control  access,  and  create  reports. 

Key  to  the  MIS  are  the  data  elements  that  reside  in  the  database.  Listed  below  are  the 
types  of  risk  information  that  will  be  included  in  the  database.  "Element"  is  the  title  of  the 
database  field;  "Description"  is  a  summary  of  the  field  contents.  The  Risk  Management 
Coordinator  will  create  the  standard  reports  such  as,  the  RIF,  Risk  Monitoring,  etc.  The 
RMIS  also  has  the  ability  to  create  "ad  hoc"  reports,  which  can  be  designed  by  users  and 
the  Risk  Management  Coordinator. 
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Element 

Description 

Risk  Identification 
(ID)  Number 

Identifies  the  risk  and  is  a  critical  element  of  information,  assuming  that  a 
relational  database  will  be  used  by  the  PMO.  (Construct  the  ID  number  to 
identify  the  organization  responsible  for  oversight.) 

Risk  Event 

States  the  risk  event  and  identifies  it  with  a  descriptive  name.  The  statement 
and  risk  identification  number  will  always  be  associated  in  any  report. 

Priority 

Reflects  the  importance  of  this  risk  priority  assigned  by  the  PMO  compared  to 
all  other  risks,  e.g.,  a  one  (1)  indicates  the  highest  priority. 

Data  Submitted 

Gives  the  date  that  the  RIF  was  submitted. 

Major  System/ 
Component 

Identifies  the  major  system/component  based  on  the  WBS. 

Subsystem/ 
Functional  Area 

Identifies  the  pertinent  subsystem  or  component  based  on  the  WBS. 

Category 

Identifies  the  risk  as  technical/performance  cost  or  schedule  or  combination  ot 
these. 

Statement  of  Risk 

Gives  a  concise  statement  (one  or  two  sentences)  or  the  risk. 

Description  of 

Risk 

Briefly  describes  the  risk.  Lists  the  key  processes  that  are  involved  in  the 
design,  development,  and  production  of  the  particular  system  or  subsystem.  If 
technical/performance,  includes  how  it  is  manifested  (e.g.,  design  and 
engineering,  manufacturing,  etc.) 

Key 

Parameters 

Identifies  the  key  parameter,  minimum  acceptable  value,  and  goal  value,  if 
appropriate.  Identifies  associated  subsystem  values  required  to  meet  the 
minimum  acceptable  value  and  describes  the  principal  events  planned  to 
demonstrate  that  the  minimum  value  has  been  met. 

Assessment 

States  if  an  assessment  has  been  done.  Cites  the  Risk  Assessment  Report,  if 
appropriate. 

Analyses 

Briefly  describes  the  analysis  done  to  assess  the  risk.  Includes  rationale  and 
basis  for  results. 

Probability  of 
Occurrence 

States  the  likelihood  of  the  event  occurring,  based  on  definitions  in  the 
program’s  Risk  Management  Plan. 

Consequence 

States  the  consequence  of  the  event,  if  it  occurs,  based  on  definitions  in  the 
program’s  Risk  Management  Plan. 

Time  Sensitivity 

Estimates  the  relative  urqency  for  implementing  the  risk-handling  option. 

Other  Affected 

Areas 

If  appropriate,  identifies  any  other  subsystem  or  process  that  this  risk  affects. 

Risk  Handling  Plans 

Briefly  describes  plans  to  mitigate  the  risk.  Refers  to  any  detailed  plans  that 
may  exist,  if  appropriate. 

Risk  Monitoring 
Activity 

Measures  using  metrics  for  tracking  progress  in  implementing  risk  handling 
plans  and  achieving  planned  results  for  risk  reduction. 

Status 

Briefly  reports  the  status  of  the  risk-handling  activities  and  outcomes  relevant 
to  any  risk  handling  milestones. 

Status  Due  Date 

Lists  date  of  the  status  report. 

Assignment 

Lists  individual  assigned  responsibility  for  mitigation  activities. 

Reported  By 

Records  name  and  phone  number  of  individual  who  reported  the  risk. 

DBMS  Elements 
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Risk  Information  Form 

Risk  Identification  Number 

Risk  Event: 

Priority 

Date 

Major  System/Component/Functional  Area: 

Category: 

Statement  of  Risk: 

Description  of  Risk: 

Key  Parameters: 

Assessment: 

Analysis: 

Process  Variance 

Probability  of  Occurrence: 

Consequence: 

Time  Sensitivity: 

Other  Affected  Areas: 

Risk  Handling  Plans:  < 

Risk  Monitoring  Activity: 

Status 

Status  Date: 

Assignment: 

Reported  By: 
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RISK  TRACKING  REPORT 
(EXAMPLE  REPORT) 

I.  Risk  Area  Status:  Design  Pp:  Hi  Cp:  Hi 

Significant  Design  Risks: 

1.  Title:  System  Weight  Pf:  Hi  Cp:  Hi 

Problem:  Exceed  system  weight  by  10%;  decreasing  the  range  and  increasing  fuel 
consumption. 

Action:  Examining  subsystems  to  determine  areas  where  weight  may  be  reduced.  Review¬ 
ing  the  requirement.  Closely  watching  the  effect  on  reliability  and  survivability. 

2.  Title:  Design  Analysis  Py:  Hi  Cy:  Hi 

Problem:  Failure  Modes,  Effects  and  Criticality  Analysis  (FMECA)  is  planned  too  late  to  iden¬ 
tify  and  correct  any  critical  single-point  failure  points  prior  to  design  freeze. 

Action:  Additional  resources  are  being  sought  to  expedite  performance  of  FMECA. 

II.  Risk  Area  Status:  Supportability  Pp:  Hi  Cp:  Mod/Hi 

1.  Title:  Operational  Support  Pf:  Hi  Cp:  Mod /Hi 

Problem:  Power  supply  subcontractor  is  in  financial  trouble  and  may  go  out  of  business.  No 

other  known  sources  exist. 

Action:  Doing  trade  study  to  see  if  alternative  designs  have  a  broader  power  supply  vendor 
base.  Prime  contractor  is  negotiating  with  the  subcontractor  to  buy  drawings  for  develop¬ 
ment  of  second  source. 
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Potential  Risk  Area 

Risk  Reduction  Actions 

Action  Code 

Due  Date 

Date  Completed 

Explanation 

•  Accurately 

•  Use  multiple  finite 

SE03 

31  Aug  98 

predicting  shock 

element  codes  & 

environment 

simplified  numerical 

shipboard 

models  for  early 

equipment  will 

assessments. 

experience. 

•  Shock  test  simple 

SE03 

31  Aug  99 

isolated  deck,  and 
proposed  isolated 
structure  to  improve 
confidence  in 
predictions. 

•  Evaluating 

•  Concentrate  on 

SE031 

31  Aug  98 

acoustic  impact 

acoustic  modeling 

of  the  ship 

and  scale  testing  of 

systems  that  are 

technologies  not 

not  similar  to 

demonstrated 

previous  designs. 

successfully  in  large- 
scale  tests  or  full- 

scale  trials. 

•  Factor  acoustic 

SE032 

31  Aug  99 

signature  mitigation 
from  isolated  modular 

decks  into  system 
requirements. 

Continue  model  tests 

to  validate  predictions 
for  isolated  decks. 

Watch  List  Example 
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SAMPLE  RISK  MANAGEMENT  PLAN  FOR  THE  ABC  PROGRAM  (ACAT  III,  IV) 

1.0  INTRODUCTION 

1.1  PURPOSE 

This  Risk  Management  Plan  (RMP)  presents  the  process  for  implementing  the  compre¬ 
hensive  and  proactive  management  of  risk  as  part  of  the  overall  management  of  the  ABC 
Program.  Risk  management  is  a  program  management  tool  to  handle  events  that  might 
adversely  impact  the  program,  thereby  increasing  the  likelihood  of  success.  This  RMP 
describes  a  management  tool  that  will: 

•  Serve  as  a  basis  for  identifying  alternatives  to  achieve  cost,  schedule,  and 
performance  goals, 

•  Assist  in  making  decisions  on  budget  and  funding  priorities, 

•  Provide  risk  information  for  Milestone  decisions,  and 

•  Allow  monitoring  the  health  of  the  program  as  it  proceeds. 

The  RMP  describes  methods  for  assessing  (identifying  and  analyzing),  prioritizing,  and 
monitoring  risk  drivers;  developing  risk-handling  approaches,  and  applying  adequate 
resources  to  handle  risk.  It  assigns  specific  responsibilities  for  these  functions,  and  pre¬ 
scribes  the  documenting,  monitoring,  and  reporting  processes  to  be  followed. 

If  necessary,  this  RMP  will  be  updated  on  the  following  occasions:  (1)  whenever  the  ac¬ 
quisition  strategy  changes,  or  there  is  a  major  change  in  program  emphasis;  (2)  in  prepa¬ 
ration  for  major  decision  points;  (3)  in  preparation  for,  and  immediately  following,  tech¬ 
nical  audits  and  reviews;  (4)  concurrent  with  the  review  and  update  of  other  program 
plans;  and  (5)  in  preparation  for  a  POM  submission. 

2.0  PROGRAM  SUMMARY 

2.1  DESCRIPTION 

The  ABC  Program  is  an  ACAT  III  level  program  that  was  initiated  in  response  to  the  NEW 
COM  Operational  Requirements  Document  (ORD)  XXX,  dated  DD-MM-YYYY.  The 
program  will  provide  an  ABC  communications  system  that  will  be  the  common  system 
(transmitter/receiver/controller)  for  all  DoD  components  for  UHF  satellite 
communications.  All  DoD  systems  requiring  UHF  satellite  communications  procured 
subsequent  to  Initial  Operational  Capability  (IOC)  of  the  ABC  system  will  incorporate  it 
to  meet  their  needs.  The  Bx  Unmanned  Air  Vehicle  is  the  lead  system  for  integration.  The 
program  has  completed  the  Program  Definition  and  Risk  Reduction  phase  and  is  preparing 
for  a  Milestone  II  decisions. 

The  system  will  be  acquired  using  off-the-shelf  UHF  satellite  communications  systems. 
During  Phase  I  of  the  program,  two  contractors  delivered  prototypes  of  their  systems. 
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One  is  a  ruggedized  commercial  product  and  the  other  is  built  to  military  specifications. 
The  Government  tested  both  systems  against  functional  and  performance  requirements 
and  some  environmental  extremes.  Although,  each  failed  portions  of  the  tests,  both  were 
evaluated  as  mature  enough  to  represent  an  acceptable  risk  for  proceeding  to  Phase  II  of 
the  program,  Engineering  and  Manufacturing  Development. 

2.2  ACQUISITION  STRATEGY 

The  Government  will  invite  the  contractors  that  participated  in  Phase  I  of  the  program 
to  submit  proposals  to  refine  their  approached  into  a  stable,  interoperable,  producible, 
supportable,  and  cost-effective  design;  validate  the  manufacturing  or  production  pro¬ 
cess;  and  demonstrate  system  capabilities  through  testing.  The  Government  will  select 
one  of  the  two  proposals  for  Phase  II  of  the  program.  The  contractor,  upon  demonstra¬ 
tion  of  exit  criteria  (See  Annex  A),  will  proceed  with  a  Low  Rate  Initial  Production  (LRIP) 
of  the  system. 

The  IOC  (20  systems)  for  the  ABC  system  is  required  by  FY-02  to  support  the  fielding  of 
the  Bx  UAV.  Production  capacity  for  the  ABC  system  at  IOC  is  expected  to  be  20  units  per 
month  to  meet  the  demand  of  new  systems. 

2.3  PROGRAM  MANAGEMENT  APPROACH 

The  ABC  Program  Manager  (PM)  reports  to  the  Program  Director,  Satellite  Communica¬ 
tions  who  has  responsibility  for  all  satellite  communications  systems.  The  ABC  Pro¬ 
gram  Office  (PO)  is  composed  of  the  PM  and  one  assistant,  with  matrix  support  from 
the  systems  command  organizations,  and  program  management  support  from  an  exter¬ 
nal  contractor.  An  integrated  management  approach  will  be  used  for  this  program.  The 
government  and  selected  contractor  will  have  representation  on  Integrated  Product 
Teams  (IPTs)  that  will  focus  on  cost,  design,  test,  manufacturing,  and  support  of  the 
system.  The  PM  chairs  the  government  IPT  that  develops  strategies  for  acquisition  and 
contracts. 

3.0  RISK-RELATED  DEFINITIONS 

The  Defense  Acquisition  Deskbook  (DAD)  section  2521  contains  the  definitions  for  risk,  risk 
management,  risk  events,  and  the  terms  associated  with  risk  management  that  will  be 
used  by  the  ABC  PO.  Variation  and  clarification  of  definitions  that  appear  in  the  DAD,  as 
they  are  used  in  the  ABC  program  are  described  below. 

3.1  TECHNICAL  RISK 

This  is  the  risk  associated  with  the  evolution  of  the  design,  production,  and  supportability 
of  the  ABC  system  affecting  the  level  of  performance  necessary  to  meet  the  operational 
requirements.  The  contractor  and  subcontractors'  design,  test,  and  production  processes 
(process  risk)  influence  the  technical  risk  and  the  nature  of  the  product  as  depicted  in  the 
various  levels  of  the  Work  Breakdown  Structure  (product  risk).  Process  risks  are  assessed 
in  terms  of  process  variance  fro  known  best  practices  and  potential  consequences  of  the 
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variance.  Product  risks  are  assessed  in  terms  of  technical  performance  measures  and  ob¬ 
served  variances  from  established  profiles. 

3.2  COST  RISK 

The  risk  associated  with  the  ability  of  the  program  to  achieve  its  life-cycle  cost  objectives. 
Two  risk  areas  bearing  on  cost  are  (1)  the  risk  that  the  cost  estimates  and  objectives  are 
accurate  and  reasonable  and  (2)  the  risk  that  program  execution  will  not  meet  the  cost 
objectives  as  a  result  of  a  failure  to  mitigate  technical  risks. 

3.3  RISK  RATINGS 

This  is  the  value  that  is  given  to  a  risk  event  (or  the  program  overall)  based  on  the  analysis 
of  the  likelihood/probability  and  consequences  of  the  event.  For  the  ABC  program,  risk 
ratings  of  Low,  Moderate,  or  High  will  be  assigned  based  on  the  following  criteria.  See 
Section  6.2  give  guidance  on  determining  likelihood  and  consequences  and  defines  the 
criteria. 

4.0  RISK  MANAGEMENT  STATUS  AND  STRATEGY 

4.1  RISK  MANAGEMENT  STATUS 

As  a  result  of  the  Program  Definition  and  Risk  Reduction  Phase,  the  overall  risk  of  the 
ABC  Program  for  Milestone  II  is  assessed  as  moderate,  but  acceptable.  Moderate  risk 
functional  areas  are  environmental  requirements;  form,  fit  and  function;  integration;  manu¬ 
facturing;  and  cost. 

4.2  RISK  MANAGEMENT  STRATEGY 

The  ABC  Program  risk  management  strategy  is  to  handle  program  risks,  both  technical 
and  non-technical,  before  they  become  problems,  causing  serious  cost,  schedule,  or  per¬ 
formance  impacts.  This  strategy  is  an  integral  part  of  the  Acquisition  Strategy  and  the 
program  management  approach,  and  will  be  executed  primarily  through  the  Government- 
Contractor  PIPT  organization.  The  PIPTs  will  continuously  and  proactively  assess  critical 
areas  (especially  those  listed  in  the  previous  paragraph)  to  identify  and  analyze  specific 
risks  and  will  develop  options  to  mitigate  all  risks  designated  as  moderate  and  high.'  The 
PIPTs  will  also  identify  the  resources  required  to  implement  the  developed  risk-handling 
options.  The  PM,  through  the  IIPT  (Integrating  Integrated  Product  Team),  will  review 
and  approve  the  PIPT  options.  Once  approved,  the  options  will  be  incorporated  into  the 
program  integrated  master  plan  (IMP)  and  integrated  master  schedule  (IMS).  The  PIPTs 
will  monitor  the  effectiveness  of  the  selected  handling  options,  and  adjust  the  risk  han¬ 
dling  approach  as  necessary. 

IPTs  will  keep  risk  information  current  by  using  the  risk  management  information  system 
described  in  paragraph  6.5.  Risk  status  will  be  reported  at  all  program  reviews.  As  new 
information  becomes  available,  the  PO  and  contractor  will  conduct  additional  reviews  to 
ascertain  if  new  risks  exit.  The  goal  is  to  be  continuously  looking  to  the  future  for  areas 
that  may  severely  impact  the  program. 
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5.0  RISK  MANAGEMENT  ORGANIZATION 


5.1  PROGRAM  OFFICE 

The  ABC  Program  risk  management  organization  is  shown  in  Figure  5-1 .  This  structure  is 
integrated  into  the  contractor  and  Government's  existing  organizations.  Program  Inte¬ 
grated  Product  Teams  (PIPTs)  will  be  formed  for  the  functional  areas  that  are  critical  to  the 
success  of  the  program.  All  functional  areas  not  covered  by  a  PIPT  will  be  assessed  and 
reviewed  by  the  program  Integrating  Integrated  Product  Team  (IIPT)  co-chaired  by  the 
ABC  PM  and  contractor  PM,  to  ensure  adequate  vigilance  against  emerging  risk  areas. 
Independent  risk  assessors  amy  conduct  reviews,  when  directed  by  the  PM,  to  ensure  the 
interface  requirements  of  user  systems  are  being  met  by  the  ABC  system  design. 


Figure  5-1.  ABC  Risk  Management  Organization 


The  PM  the  is  overall  coordinator  of  Risk  Management  Program  and  is  responsible  for: 

•  Maintaining  this  Risk  Management  Plan 

•  Maintaining  the  Risk  Management  Database 

•  Approving  risk-handling  options 

•  Incorporating  risk-handling  actions  into  the  program  master  plan  and  schedule 

•  Briefing  the  decision  makers  on  the  status  of  ABC  Program  risk  efforts 

•  Preparing  risk  briefings,  reports,  and  documents  required  for  Program  Reviews 
and  the  acquisition  Milestone  decision  processes. 

IIPT 

The  IIPT  is  responsible  for  complying  with  the  DoD  risk  management  policy  and  for  struc¬ 
turing  an  efficient  and  useful  ABC  risk  management  approach  and  supporting  the  Risk 
Management  Coordinator/ PM  in  carrying  out  his  responsibilities.  The  PM  and  contractor 
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PM  Co-Chair  the  IIPT.  The  IIPT  membership  may  be  adjusted,  but  is  initially  established 
as  the  chairs  of  the  PIPTs,  a  representative  from  the  joint  requirements  and  users'  office, 
and  a  representative  from  the  contractor. 

PIPTs 

The  program  PIPTs  are  the  backbone  of  the  program  risk  management  efforts.  They  will 
execute  the  following  responsibilities  relative  to  their  functional  areas: 

•  Conduct  risk  assessments  and  develop  risk-handling  options,  to  include  mitiga¬ 
tion  plans  and  resources  required. 

•  Monitor  effectiveness  of  risk-handling  actions. 

•  Review  and  recommend  to  the  PM  changes  in  the  overall  risk  management  ap¬ 
proach  based  on  lessons  learned. 

•  Update  the  risk  assessments  quarterly,  or  as  directed. 

•  Ensure  information  in  the  Risk  Management  Database  is  current. 

•  Prepare  risk  status  reports  in  their  areas  for  all  Program  and  Design  Reviews. 

•  Ensure  Design/Build  Team  responsibilities  incorporate  appropriate  risk  manage¬ 
ment  tasks. 

•  Coordinate  PIPT  risk  management  activities  with  the  IIPT. 

6.0  RISK  MANAGEMENT  STRUCTURE  AND  PROCEDURES 

The  ABC  program  will  use  a  structured  risk  management  approach  consisting  of  four 
elements:  planning,  assessment,  handling,  and  monitoring.  These  elements  and  the  gen¬ 
eral  procedures  to  be  used  for  each  of  them  are  described  in  subsequent  paragraphs  of  this 
section.  A  number  of  guidance  documents  are  useful  in  addressing  these  risk  manage¬ 
ment  elements,  and  should  be  used  as  appropriate  by  each  PIPT.  Some  of  these  docu¬ 
ments  are  listed  below.  (This  list  is  not  meant  to  be  complete.) 

•  Defense  Acquisition  Deskbook- Section  2.5.2,  Risk  Management 

•  DSMC,  Risk  Management  Guide ,  March  1998 

•  AFMC  Pamphlet  63-101,  Risk  Management,  9  July  1997 

•  The  Navy’s  Best  Practices  Manual ,  NAVSO  P-6071,  and  Top  Eleven  Ways  to  Manage 
Technical  Risk,  NAVSO  P-3686,  provide  insight  into  best  practices. 
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6.1  RISK  PLANNING 


Risk  planning  is  essential  for  the  execution  of  a  successful  risk  management  program.  It 
will  be  done  continuously  by  all  PIPTs  as  an  integral  part  of  normal  ABC  program  man¬ 
agement.  This  RMP  serves  as  the  basis  for  all  detailed  risk  planning,  which  must  be  con¬ 
tinuous.  The  following  paragraphs  provide  direction  for  the  PIPTs  on  the  conduct  of  risk 
planning  for  this  program. 

•  PIPTs  will  develop  an  organized  and  thorough  approach  to  assess,  handle,  and 
monitor  risks.  It  will  assign  responsibilities  for  specific  risk  management  actions 
and  establish  internal  risk  reporting  and  documentation  requirements.  The  IIPT 
will  monitor  the  planning  activities  of  the  PIPTs  to  ensure  that  they  are  consistent 
with  this  RMP  and  that  appropriate  revisions  to  this  plan  are  made  when  required 
to  reflect  significant  changes  resulting  from  the  PIPT  planning  efforts. 

•  Each  PIPT  will  establish  metrics  that  will  measure  the  effectiveness  of  their  planned 
risk-handling  options.  See  Annex  C  for  an  example  of  metrics  that  may  be  used. 

•  Each  PIPT  will  identify  the  resources  required  to  implement  the  risk  management 
actions.  These  resources  include  time,  material,  personnel,  and  cost.  Training  is  a 
major  consideration.  All  PIPT  members  should  receive  instruction  on  the  funda¬ 
mentals  of  risk  management  and  special  training  in  their  areas  of  responsibility,  if 
necessary.  General  risk  management  training  will  be  arranged  by  the  PO;  PIPT 
leaders  will  identify  any  specialized  training  needs. 

•  This  RMP  establishes  the  basic  documentation  and  reporting  requirements  for  the 
program.  PIPTs  should  identify  any  additional  requirements,  consistent  with  this 
RMP,  that  might  be  needed  to  effectively  manage  risk  at  their  level. 

6.2  RISK  ASSESSMENT 

The  risk  assessment  process  includes  the  identification  of  critical  risk  events/ processes, 
the  analyses  of  these  events/processes  to  determine  the  likelihood  of  occurrence/ process 
variance  and  consequences,  and  the  priority  of  the  risks.  The  output  of  this  process  pro¬ 
vides  the  foundation  for  all  the  program  risk-handling  actions.  Therefore,  it  is  essential 
that  all  members  of  the  ABC  program  team  be  as  thorough  as  possible  when  identifying 
and  analyzing  risks.  In  addition  to  the  normal  areas  of  design ,  test,  manufacturing,  etc., 
PIPTs  must  identify  and  analyze  the  risks  associated  with  such  areas  as  manpower,  envi¬ 
ronmental  impact,  system  safety  and  health  analysis,  and  security  considerations.  The 
Defense  Acquisition  Deskbook,  Section  2524,  provides  information  on  various  risk  assess¬ 
ment  techniques. 

Risk  assessments  should  be  done  by  the  PIPTs  and  the  IIPT  with  active  participation  of 
both  Government  and  contractor  personnel.  When  necessary  or  appropriate,  the  PIPTs 
and  the  IIPT  can  direct  a  contractor-only  assessment,  or  conduct  a  Government  assess¬ 
ment.  PIPTs  and  the  IIPT  should  continually  assess  the  risks  in  their  areas,  reviewing 
critical  risk  areas,  risk  ratings  and  prioritization,  and  the  effectiveness  of  risk-mitigation 
actions  whenever  necessary  to  assess  progress.  The  assessment  process  will  be  iterative. 
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with  each  assessment  building  on  the  results  of  previous  assessments.  PIPTs  and  the  IIPT 
will  use  the  current  assessment  baseline  as  the  starting  point  for  their  initial  assessment 
during  this  phase.  This  baseline  is  a  combination  of  the  risk  assessment  delivered  by  the 
contractors  as  part  of  Phase  0,  the  PMO  process  risk  assessment  done  before  Milestone  I, 
and  the  post  award  Integrated  Baseline  Review  (IBR).  Risk  assessments  will  be  updated 
and  the  results  presented  at  all  functional  and  program  reviews,  with  a  final  update  for 
this  phase  prepared  not  later  than  six  months  prior  to  the  next  scheduled  Milestone  decision. 

6.2.1  Risk  Identification 

Each  PIPT  will  review  all  aspects  of  their  functional  areas  to  determine  the  critical  events 
that  would  prevent  the  program  from  achieving  its  objectives.  They  should  apply  the 
knowledge,  best  judgment  and  experience  of  the  PIPT  members,  lessons  learned  from 
similar  programs,  and  the  opinion  of  subject-matter  experts  (SMEs)  to  identify  these  risk 
events.  PIPTs  should  follow  these  general  procedures  to  identify  risk  events: 

•  Understand  the  requirements  and  the  program  performance  goals,  which  are  de¬ 
fined  as  thresholds  and  objectives  (see  DoD  5000.2-R).  Understand  the  operational 
(functional  and  environmental)  conditions  under  which  the  values  must  be  achieved 
as  described  in  the  Design  Reference  Mission  Profile.  The  ORD  and  Acquisition 
Program  Baseline  (APB)  contain  Key  Performance  Parameters  (KPPs). 

•  Determine  technical /performance  risks  related  to  engineering  and  manufacturing 
processes.  Identify  those  processes  that  are  planned  or  needed  to  design,  develop, 
produce,  and  support  the  system.  Compare  these  processes  with  industry  best  prac¬ 
tices  and  identify  any  variances  or  new,  untried  processes.  These  variances  or  un¬ 
tried  practices  are  sources  of  risk.  The  contractor  should  review  the  processes  to  be 
used  by  its  subcontractors  to  ensure  they  are  consistent  with  best  industry  prac¬ 
tices.  Table  4-2  of  the  DSMC  Risk  Management  Guide  shows  some  of  the  specific  of 
sources  of  process  risk,  and  should  be  used  by  the  PIPTs.  NAVSO  P-6071,  Best  Prac¬ 
tices,  which  describes  risks  associated  with  design,  test,  production,  facilities,  lo¬ 
gistics,  management,  and  funding,  should  also  be  used  by  the  PIPTs  to  identify 
risks. 

•  Determine  technical  /performance  risks  associated  with  the  product  (the  ABC  com¬ 
munications  system)  in  the  following  critical  risk  areas:  design  and  engineering, 
technology,  logistics,  concurrency,  and  manufacturing.  The  design  and  manufac¬ 
turing  PIPTs  will  identify  the  contract  WBS  elements  down  to  level  3,  and  evaluate 
each  of  these  elements  to  identify  risk  events.  They  will  use  a  variety  of  methods  to 
accomplish  this:  review  of  similar  programs,  existing  program  plans,  expert  opin¬ 
ion,  etc. 

•  Identify  schedule  risk.  Each  PIPT  will  determine  the  schedule  risk  associated  with 
its  functional  area.  When  identifying  this  schedule  risk,  they  will  consider  the  risk 
that  the  schedule  estimate  is  accurate,  and  the  risk  that  the  established  schedule 
can  be  met.  The  IIPT  will  monitor  the  development  of  the  schedule  risk  in  each 
PIPT,  and  consolidate  these  risks  to  identify  overall  program  schedule  risk. 
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•  Identify  cost  risk.  Each  PIPT  will  determine  the  cost  risk  associated  with  its  func¬ 
tional  area.  They  will  identify  risks  associated  with  the  accuracy  of  the  cost  esti¬ 
mates  developed  for  their  areas,  and  the  risk  that  the  established  cost  objectives 
will  be  met.  The  Cost  PIPT  will  monitor  the  development  of  the  other  PIPT  cost 
risk  efforts,  and  consolidate  their  risks  into  a  set  of  overall  program  cost  risks. 

•  All  identified  risks  will  be  documented  in  the  RMIS,  with  a  statement  of  the  risk 
and  a  description  of  the  conditions  or  situations  causing  concern  and  the  context  of 
the  risk.  See  Paragraph  6.4  for  guidance  on  documenting  identified  risks. 

In  identifying  risks,  PIPTs  should  be  particularly  alert  for  the  following  indicators.  They 
are  common  sources  of  risk  for  all  programs,  and  will  be  applicable  to  the  ABC  program. 

•  Requirements  that  are  not  clearly  stated  or  stable, 

•  Failure  to  use  Best  Practices, 

•  Use  of  new  processes  materials,  or  applications  of  existing  technologies, 

•  Use  of  processes  lacking  rigor  in  terms  of  maturity,  documentation  of  established 
procedures,  and  validation, 

•  Insufficient  resources — the  people,  funds,  schedule,  and  tools,  necessary  for  suc¬ 
cessful  development,  test,  production  and  support  of  the  ABC  program, 

•  Lack  of  a  formalized  failure,  reporting,  analyze,  and  corrective  action  (FRACAS) 
system, 

•  Use  of  suppliers  or  subcontractors  who  are  inexperienced  in  the  processes  for  de¬ 
signing  and  producing  required  products, 

•  Failure  of  prime  contractor  to  effectively  monitor  processes  and  establish  qual¬ 
ity  requirements  for  suppliers  and  subcontractors. 

6.2.2  Risk  Analysis 

Risk  Analysis  is  an  evaluation  of  the  identified  risk  events  to  determine  the  likelihood  of 
the  events  occurring  and  their  consequences,  to  assign  a  risk  rating  based  on  the  program 
criteria,  and  to  prioritize  the  risks.  Each  PIPT  and  the  IIPT  are  responsible  for  analyzing 
those  risk  events  they  identify.  They  may  use  subject  matter  experts  for  assistance,  such  as 
Field  Activities,  Service  Laboratories,  contractors,  or  outside  consultants.  The  use  of  ex¬ 
ternal  assets  will  be  coordinated  through  the  PMO.  The  results  of  the  analysis  of  all  iden¬ 
tified  risks  must  be  documented  in  the  RMIS. 

There  are  a  number  of  techniques  available  to  support  risk  analysis,  to  include  studies, 
test  results,  modeling  and  simulation,  and  the  opinions  of  qualified  experts  (to  include 
justification  of  their  judgment).  The  DAD,  Section  2524.2  describes  a  number  of  analysis 
techniques  that  may  be  useful.  Regardless  of  the  technique  used,  PIPTs  and  the  IIPT  will 
identify  all  assumptions  made  in  analyzing  risk  and,  where  appropriate,  conduct  a 
sensitivity  analysis  of  assumptions. 
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For  each  risk  event,  the  following  risk  analysis  guidelines  will  be  used: 

*  Likelihood/Probability 

For  each  risk  identified,  determine  the  likelihood  that  the  event  will  occur.  Five  levels  of 
likelihood  will  be  used  for  the  ABC  program.  Table  6-1  shows  these  levels  and  their  defi¬ 
nitions.  PIPTs  and  the  IIPT  will  assign  one  of  these  values  to  each  identified  risk  event 
based  on  their  analysis  of  the  event.  For  example,  if  it  is  known  that  there  will  be  a  vari¬ 
ance  between  the  soldering  process  to  be  used  for  component  X  and  the  industry  stan¬ 
dard,  this  process  variance  risk  event  will  be  assigned  a  likelihood  value  of  e  near. 
Similarly,  if  the  Manufacturing  PIPT  determines  that  the  schedule  estimate  for  the  fabrica¬ 
tion  of  component  Y  is  overly  optimistic,  and  will  probably  not  be  attained,  it  would  as¬ 
sign  a  likelihood  level  of  "c"  or  “d"  depending  on  its  analysis  of  the  schedule  estimate. 


Level 

Likelihood  of  Occurrence 

a 

Remote 

b 

Unlikely 

c 

Likely 

d 

Highly  likely 

e 

Near  certainty 

Table  6-1.  Likelihood  Levels 


•  Consequence 

For  each  risk  identified,  the  following  question  must  be  answered:  Given  the  event  occurs , 
what  is  the  magnitude  of  the  consequence?  For  the  ABC  program,  consequence  will  be  deter¬ 
mined  in  each  of  four  areas:  technical  performance,  schedule,  cost,  and  impact  on  other 
teams. 

Technical  Performance:  This  category  relates  to  the  risks  associated  with  the  pro¬ 
cesses  to  be  used  in  the  development,  testing,  and  manufacturing  of  the  ABC  sys¬ 
tem,  and  the  nature  of  the  ABC  communications  system.  It  includes  the  form,  fit, 
function,  manufacturability,  supportability,  etc.  Essentially,  technical  risk  includes 
all  requirements  that  are  not  part  of  cost  and  schedule.  The  wording  of  each  conse¬ 
quence  level  is  oriented  toward  design  and  production  processes,  life  cycle  sup¬ 
port,  and  retirement  of  the  system.  For  example,  the  word  "margin"  could  apply 
to  weight  margin  during  design,  safety  margin  during  testing,  or  machine  perfor¬ 
mance  margin  during  production. 

Schedule:  The  description  in  the  Schedule  is  self-explanatory.  The  need  dates,  key 
milestones,  critical  path,  and  key  team  milestones  are  meant  to  apply  to  all  pro¬ 
gram  areas  and  PIPTs. 

Cost:  Since  costs  vary  from  component  to  component  and  process  to  process,  the 
percentage  criteria  shown  in  the  figure  may  not  strictly  apply  at  the  lower  levels  of 
the  WBS.  PIPT  and  IIPT  leaders  may  set  the  percentage  criteria  that  best  reflect 
their  situation.  Flowever,  when  costs  are  rolled  up  at  higher  levels  (e.g.,  Program), 
the  definitions  shown  will  be  used. 
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Impact  on  Other  Teams:  Both  the  consequences  of  a  risk  and  the  mitigation  actions 
associated  with  handling  the  risk  may  impact  another  team.  This  may  involve  addi¬ 
tional  coordination  or  management  attention  (resources),  and  may  therefore  increase 
the  level  of  risk.  This  is  especially  true  of  mitigation  actions  that  involve  the  use  of 
common  manufacturing  processes  and/or  equipment. 


PIPTs  and  the  IIPT  will  evaluate  each  risk  event  in  terms  of  these  areas,  and  assign  a  level 
of  consequence  (1-5).  Table  6-2  shows  these  5  levels  of  consequence,  and  defines  the  levels 
for  each  area.  This  table  will  be  used  when  assigning  the  consequence  magnitude. 


Level 

Technical 

Performance 

Schedule 

Cost 

Impact  on 

Other  Teams 

1 

Minimal  or  no  impact 

Minimal  or  no  impact 

Minimal  or 
no  impact 

None 

2 

Acceptable  with  some 
reduction  in  margin 

Additional  resources 
required.  Able  to  meet 
need  dates 

<5% 

Some  impact 

3 

Acceptable  with 
significant  reduction 
in  margin 

Minor  slip  in  key  milestone. 
Not  able  to  meet  need  dates 

5-7% 

Moderate 

impact 

m 

Acceptable— no 
remaining  margin 

Major  slip  in  key  milestone 
or  critical  path  impacted 

7-10% 

Major  impact 

5 

Unacceptable 

Can’t  achieve  key  team  or 
major  program  milestone 

>10% 

Unacceptable 

Table  6-2.  Risk  Consequence 


6.2.3  Risk  Rating 

Each  identified  risk  will  be  assigned  a  risk  rating  based  on  the  joint  consideration  of  event 
likelihood  and  consequence.  This  rating  is  a  reflection  of  the  severity  of  the  risk  and 
provides  a  starting  point  for  the  development  of  options  to  handle  the  risk.  It  is  important 
to  consider  both  the  likelihood  and  consequences  in  establishing  the  rating,  for  there  may 
be  risk  events  that  have  a  low  likelihood,  but  whose  consequences  are  so  severe  that  the 
occurrence  of  the  event  would  be  disastrous  to  the  program. 

Figure  6-1  describes  the  risk  rating  process  that  will  be  used  in  this  program.  PIPTs  and 
the  IIPT  will  analyze  each  risk  event  to  determine  the  likelihood  and  consequence  values 
using  the  definitions  in  Tables  6-1  and  6-2;  they  will  determine  the  consequence  for  each  of 
the  four  areas  (technical  performance,  schedule,  cost,  and  team  impact).  The  values  will 
be  used  to  determine  the  risk  rating  using  the  Assessment  Guide  in  Figure  6-1.  The  As¬ 
sessment  Guide  defines  the  risk  rating  associated  with  each  combination  of  likelihood 
and  consequence  values,  and  will  be  used  throughout  the  program.  For  example,  conse¬ 
quence  /likelihood  level  lb  corresponds  to  a  risk  rating  of  (L)  LOW,  level  4b  corresponds 
to  MODERATE  risk,  and  level  5c  corresponds  to  HIGH  risk. 

Those  risk  events  that  are  assessed  as  MODERATE  or  HIGH  will  be  submitted  to  the  ABC 
PM  on  a  Risk  Identification  Form  (RIF).  See  Appendix  B  for  the  RIF  format.  PIPTs  and  the 
IIPT  must  actively  manage  these  MODERATE  and  HIGH  risks.  They  must  also  continu- 
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ously  assess  the  other  identified  risks  in  their  areas  to  see  if  their  ratings  have  become 
MODERATE  or  HIGH. 

6.2.4  Risk  Prioritization 

PIPTs  and  the  IIPT  will  prioritize  the  MODERATE  and  HIGH  risks  in  their  areas.  This 
prioritization  will  provide  the  basis  for  the  development  of  risk  handling  plans  and  the 
allocation  of  risk  management  resources.  Prioritization  will  be  accomplished  using  ex¬ 
pert  opinion  within  the  PIPTs,  and  will  be  based  on  the  following  criteria: 

•  Risk  Rating— Obviously  HIGH-MODERATE. 

•  Consequence— Within  each  rating,  the  highest  value  of  consequence,  e.g.,  e. 

•  Urgency — How  much  time  is  available  before  risk-handling  actions  must  be 
initiated. 


Likelihood — Within  each  rating,  the  highest  value,  e.g.,  "e. 


Level 

Likelihood  of  Occurrence 

Remote 

Unlikely 

1 

Likely 

Highly  likely 

1 

e 

Near  certainty 

Risk  Assessment  Process 


ASSESSMENT  GUIDE 


w 


e 

D 

O 

□ 

a 

m 

D 

□ 

o 

□ 

qI 

Ic 

D 

Q 

□ 

Q 

D 

!b 

a 

D 

a 

a 

HI 

a 

D 

D 

n 

a 

m 

i 

2 

3 

4 

5 

Consequence 


RISK  ASSESSMENT 

HIGH— Unacceptable,  Major 
disruption  likely.  Different 
approach  required.  Priority 
management  attention  required. 

MODERATE— Some  disruption. 
Different  approach  may  be 
required.  Additional 
management  attention  may  be 
needed. 

LOW— Minimum  impact. 
Minimum  oversight  needed  to 
ensure  risk  remains  low. 


Level 

Technical 

Performance 

Schedule 

Cost 

Impact  on 

Other  Teams 

a 

Minimal  or  no  impact 

Minimal  or  no  impact 

Minimal  or 
no  impact 

None 

■ 

Acceptable  with  some 
reduction  in  margin 

Additional  resources 
required.  Able  to  meet 
need  dates 

<5% 

Some  impact 

c 

Acceptable  with 
significant  reduction 
in  margin 

Minor  slip  in  key  milestone. 
Not  able  to  meet  need  dates 

5-7% 

Moderate 

impact 

H 

Acceptable— no 
remaining  margin 

Major  slip  in  key  milestone 
or  critical  path  impacted 

7-10% 

Major  impact 

Unacceptable 

Can’t  achieve  key  team  or 
major  program  milestone 

>10% 

Unacceptable 

Figure  6-1.  Risk  Assessment  Process 
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The  IIPT  will  review  the  prioritized  list  of  PIPT-developed  risks,  and  integrate  them  into  a 
single  list  of  prioritized  program  risks,  using  the  same  criteria. 

6.3  RISK  HANDLING 

After  the  program's  risks  have  been  identified,  analyzed,  and  prioritized,  PIPTs  and  the 
IIPT  must  develop  an  approach  for  handling  each  MODERATE  and  HIGH  risk.  For  all 
such  risks,  the  various  handling  techniques  should  be  evaluated  in  terms  of  feasibility, 
expected  effectiveness,  cost  and  schedule  implications,  and  the  effect  on  the  system's  tech¬ 
nical  performance,  and  the  most  suitable  technique  selected.  The  DAD  section  2524.3 
contains  information  on  the  risk-handling  techniques  and  various  actions  that  can  be  used 
to  implement  them.  Reducing  requirements  as  a  risk  avoidance  technique  will  be  used 
only  as  a  last  resort,  and  then  only  with  the  participation  and  approval  of  the  user's  repre¬ 
sentative  at  the  IIPT  level. 

The  results  of  the  evaluation  and  selection  will  be  included  and  documented  in  the  RMIS 
using  the  RIF.  This  documentation  will  include  the  following  elements: 

•  What  must  be  done, 

•  List  of  all  assumptions, 

•  Level  of  effort  and  materials  required, 

•  Resources  needed  that  are  outside  the  scope  of  the  contract  or  official  tasking, 

•  Estimated  cost  to  implement  the  plan, 

•  Proposed  schedule  showing  the  proposed  start  date,  the  time  phasing  of  signifi¬ 
cant  risk  reduction  activities,  the  completion  date,  and  their  relationship  to  signifi¬ 
cant  Program  activities /milestones, 

•  Recommended  metrics  for  tracking  risk-handling  activity, 

•  Other  PIPTs,  risk  areas,  or  other  handling  plans  which  may  be  impacted, 

•  Person  responsible  for  implementing  and  tracking  the  selected  option. 

Risk  handling  actions  will  be  integrated  into  program  planning  and  scheduling,  and  in¬ 
corporated  into  the  IMP  and  IMS.  PIPTs  and  the  IIPT  will  develop  these  risk-handling 
actions  and  events  in  the  context  of  Work  Breakdown  Structure  (WBS)  elements,  estab¬ 
lishing  a  linkage  between  them  and  specific  work  packages  that  makes  it  easier  to  deter¬ 
mine  the  impact  of  actions  on  cost,  schedule,  and  performance.  The  detailed  information 
on  risk-handling  actions  and  events  will  be  included  in  the  RIF  for  each  identified  risk, 
and  thus  be  resident  in  the  RMIS. 

6.4  RISK  MONITORING 

Risk  monitoring  is  the  systematic  tracking  and  evaluation  of  the  progress  and  effective¬ 
ness  of  risk-handling  actions  by  the  comparison  of  predicted  results  of  planned  actions 
with  the  results  actually  achieved  to  determine  status  and  the  need  for  any  change  in  risk- 
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handling  actions.  The  PIPTs  and  the  IIPT  will  monitor  all  identified  risks  in  their  areas, 
with  particular  attention  to  those  rated  as  HIGH  or  MODERATE.  There  are  a  number  of 
techniques  and  tools  available  for  monitoring  the  effectiveness  of  risk-handling  actions. 
(See  DAD  section  2524.4  for  information  on  specific  techniques.)  PIPTs  and  the  IIPT  must 
select  those  that  best  suit  their  needs.  No  single  technique  or  tool  is  capable  of  providing 
a  complete  answer — a  combination  must  be  used.  At  a  minimum,  each  PIPT  and  the  IIPT 
will  use  the  Risk  Tracking  Report  (RTR)  and  Watch  List  for  day-to-day  management  and 
monitoring  of  risks.  See  Annex  B  for  examples  of  an  RTR  and  Watch  List.  The  status  of 
risk-handling  actions  for  all  MODERATE  and  HIGH  risks  will  be  an  agenda  item  at  each 
program  or  functional  area  review. 

For  each  identified  risk,  the  PIPTs  and  IIPT  will  establish  a  management  indicator  system 
(metrics)  that  provides  accurate,  timely,  and  relevant  risk  monitoring  information  in  a 
clear,  easily  understood  manner.  PIPTs  and  the  IIPT  should  select  metrics  that  portray  the 
true  state  of  the  risk  events  and  handling  actions.  See  Annex  C  for  an  example  of  metrics 
that  may  be  used. 

MODERATE  or  HIGH  risks  will  also  be  monitored  by  the  ABC  PM  through  the  IIPT,  using 
information  provided  by  the  appropriate  PIPT,  until  the  risk  is  considered  LOW  and  rec¬ 
ommended  for  "Close  Out."  PIPTs  and  the  IIPT  will  continue  to  monitor  LOW  risk  events 
in  their  areas  to  ensure  that  appropriate  risk-handling  action  can  be  initiated  if  there  are 
indications  that  the  rating  may  change. 

The  status  of  the  risks  and  the  effectiveness  of  the  risk-handling  actions  will  be  agenda 
items  for  all  functional  area  and  program  reviews,  and  will  be  reported  to  the  PM  on  the 
following  occasions: 

•  Quarterly, 

•  When  the  IPT  determines  that  the  status  of  the  risk  area  has  changed  significantly 
(as  a  minimum  when  the  risk  changes  from  high  to  moderate  to  low,  or  vice  versa), 

•  When  requested  by  the  Program  Manager. 

6.5  RISK  MANAGEMENT  INFORMATION  SYSTEM  (RMIS), 

DOCUMENTATION,  AND  REPORTS 

The  ABC  Program  uses  a  modified  version  of  Risk  Matrix  as  its  RMIS.  The  Risk  Matrix 
database  will  contain  all  of  the  information  necessary  to  satisfy  the  program  documenta¬ 
tion  and  reporting  requirements.  This  information  will  include  risk  assessment  docu¬ 
ments,  risk-handling  plans,  contract  deliverables,  if  appropriate,  and  any  other  risk-related 
reports.  The  program  office  will  use  data  from  the  RMIS  to  create  reports  for  senior  man¬ 
agement  and  for  day-to-day  management  of  the  program.  The  program  produces  a  set  of 
standard  reports  for  periodic  reporting  and  has  the  ability  to  create  ad  hoc  reports  in 
response  to  special  queries. 

Each  PIPT  and  the  IIPT  are  responsible  for  entering  and  maintaining  accurate  risk  man¬ 
agement  data  in  the  RMIS.  A  standard  format  Risk  Information  Form  (RIF)  Data  will  be 
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used  for  data  entry.  A  RIF  will  be  completed  and  submitted  when  a  potential  risk  event  is 
identified,  and  will  be  updated  as  information  becomes  available  as  the  assessment,  han¬ 
dling,  and  monitoring  functions  are  executed.  See  Annex  B  for  a  sample  of  the  RIF.  Annex 
B  also  contains  examples  of  reports  to  be  used  in  the  ABC  Program. 


ANNEX  A 

TO  ABC  RISK  MANAGEMENT  PLAN 
—CRITICAL  PROGRAM  ATTRIBUTES— 


Description 


Transmitter  Power 


Weight 


MTBF 


Receiver  Gain 


EMP  Survivability 


Heat  Dissipation 


Size 


Receiver  Range 


Transmitter  Range 


Data  Link  Operations 


Interface  Commonality 


Initial  Setup 


Identification  Time 


Accuracy  Location 


Bandwidth 


Reliabili 


Responsible  IPT  |  Remarks 


Reauirements  Stable 


Test  Plan  Approved 


Bench  Test 


Accuracy  Verified  by  Test  Data 
and  Analysis 


Toolproofing  Completed 


Logistics  Support  Reviewed  by 
User 
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ANNEX  B 

TO  ABC  RISK  MANAGEMENT  PLAN 
—MANAGEMENT  INFORMATION  SYSTEM  AND  DOCUMENTATION— 

1.0  DESCRIPTION 

In  order  to  manage  risk,  we  need  a  database  management  system  that  stores  and  allows 
retrieval  of  risk-related  data.  The  Risk  Management  Information  System  provides  data 
for  creating  reports  and  serves  as  the  repository  for  all  current  and  historical  information 
related  to  risk.  The  PM  is  responsible  for  the  overall  maintenance  of  the  RMIS,  and  he/ 
she  or  his/her  designee  are  the  only  persons  who  may  enter  data  into  the  database. 

The  RMIS  has  a  set  of  standard  reports.  If  PIPTs  or  functional  managers  need  additional 
reports,  they  should  work  with  the  PM  to  create  them.  Access  to  the  reporting  system  will 
be  controlled,  however  any  member  of  the  Government  or  contractor  team  may  obtain  a 
password  to  gain  access  to  the  information. 

In  addition  to  standard  reports,  the  PO  will  need  to  create  ad  hoc  reports  in  response  to 
special  queries,  etc.  The  PM  will  be  responsible  for  these  reports. 
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Element 

Description 

Risk  Identification 
(ID)  Number 

Identifies  the  risk  and  is  a  critical  element  of  information,  assuming  that  a 
relational  database  will  be  used  by  the  PO.  (Construct  the  ID  number  to 
identify  the  organization  responsible  for  oversight.) 

Risk  Event 

States  the  risk  event  and  identifies  it  with  a  descriptive  name.  The  statement  and 
risk  identification  number  will  always  be  associated  in  any  report. 

Priority 

Reflects  the  importance  of  this  risk  priority  assigned  by  the  PO  compared  to  all 
other  risks,  e.g.,  a  one  (1)  indicates  the  highest  priority. 

Data  Submitted 

Gives  the  date  that  the  RIF  was  submitted. 

Major  System/Com¬ 
ponent  or  Process 

Identifies  the  major  system/component  based  on  the  WBS,  or  the  process  in 
which  the  risk  event  occurs. 

Subsystem/ 
Functional  Area 

Identifies  the  pertinent  subsystem  or  component  based  on  the  WBS. 

Category 

Identifies  the  risk  as  technical/performance  cost  or  schedule  or  combination  of 
these. 

Statement  of  Risk 

Gives  a  concise  statement  (one  or  two  sentences)  or  the  risk. 

Description  of 

Risk 

Briefly  describes  the  risk;  lists  the  key  processes  that  are  involved  in  the  design, 
development,  and  production  of  the  particular  system  or  subsystem.  If  technical/ 
performance,  include  how  it  is  manifested  (e.g.,  design  and  engineering, 
manufacturing,  etc.) 

Key  parameters 

Identifies  the  key  parameter,  minimum  acceptable  value,  and  goal  value,  if 
appropriate.  Identifies  associated  subsystem  values  required  to  meet  the 
minimum  acceptable  value  and  describes  the  principal  events  planned  to 
demonstrate  that  the  minimum  value  has  been  met. 

Assessment 

States  if  an  assessment  has  been  done.  Cites  the  Risk  Assessment  Report  (see 
next  paragraph),  if  appropriate. 

Analysis 

Briefly  describes  the  analysis  done  to  assess  the  risk;  includes  rationale  and 
basis  for  results. 

Process  Variance 

States  the  variance  of  critical  technical  processes  from  known  standards  or  best 
practices,  based  on  definitions  in  the  program’s  risk  management  plan. 

Probability  of 
Occurrence 

States  the  likelihood  of  the  event  occurring,  based  on  definitions  in  the 
program’s  Risk  Management  Plan. 

Consequence 

States  the  consequence  of  the  event,  if  it  occurs,  based  on  definitions  in  the 
program’s  Risk  Management  Plan. 

Risk  Rating 

Identifies  the  rating  assigned  to  the  risk  based  on  the  criteria  established  by  the 
program. 

Time  Sensitivity 

Estimates  the  relative  urgency  for  implement  the  risk-handling  option.  If 
appropriate,  identifies  any  other  subsystem  or  process  that  this  risk  affects. 

Other  Affected 
Areas 

If  appropriate,  identifies  any  other  subsystem  or  process  that  this  risk  affects. 

Risk  Handling 

Plans 

Briefly  describes  plans  to  mitigate  the  risk.  Refers  to  any  detailed  plans  that  may 
exist,  if  appropriate. 

Risk  Monitoring 
Activity 

Measurement  and  metrics  for  tracking  progress  in  implementing  risk-handling 
plans  and  achieving  planned  results  for  risk  reduction. 

Status 

Briefly  reports  the  status  of  the  risk-handling  activities  and  outcomes  relevant 
to  any  risk  handling  milestones. 

Status  Due  Date 

Lists  date  of  the  status  report. 

Assignment 

Lists  individual  assigned  responsibility  for  mitigation  activities. 

Reported  By 

Records  name  and  phone  number  of  individual  who  reported  the  risk. 

DBMS  Elements 
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2.0  RISK  MANAGEMENT  FORMS  AND  REPORTS 

The  following  are  examples  of  basic  reports  and  forms  that  are  used  in  the  ABC  Program. 

2.1  RISK  INFORMATION  FORM 

The  PO  needs  a  document  that  serves  the  dual  purpose  of  a  source  of  data  entry  informa¬ 
tion  and  a  report  of  basic  information  for  the  PIPTs,  etc.  The  Risk  Information  Form  (RIF) 
serves  this  purpose.  It  gives  members  of  the  project  team,  both  Government  and  contrac¬ 
tors,  a  format  for  reporting  risk-related  information.  The  RIF  will  be  used  when  a  poten¬ 
tial  risk  event  is  identified  and  updated  over  time  as  information  becomes  available  and 
the  status  changes.  As  a  source  of  data  entry,  the  RIF  allows  the  database  administrator  to 
control  entries.  The  format  and  information  required  in  a  RIF  is  detailed  in  the  following 
table. 

2.2  RISK  MONITORING  DOCUMENTATION 

The  PM  needs  a  summary  document  that  tracks  the  status  of  HIGH  and  MODERATE 
risks.  The  ABC  program  will  use  a  Risk-Tracking  Report  (RTR)  that  contains  information 
that  has  been  entered  from  the  RIF.  An  example  of  the  RTR  is  shown  in  Figure  7-1 .  The  PM 
and  PIPTs  must  also  be  aware  of  upcoming  deadlines  and  events  to  ensure  they  are  not 


RISK  TRACKING  REPORT 
(EXAMPLE  REPORT) 

I.  Risk  Area  Status:  Design  Likelihood:  High  Consequence:  High 

Significant  Design  Risks: 

1.  Title:  System  Weight  Likelihood:  High  Consequence:  High 

Problem:  Exceed  system  weight  by  10%;  decreasing  the  range  and  increasing  fuel 
consumption. 

Action:  Examining  subsystems  to  determine  areas  where  weight  may  be  reduced.  Review¬ 
ing  the  requirement.  Closely  watching  the  effect  on  reliability  and  interoperability. 

2.  Title:  Design  Analysis  Likelihood:  High  Consequence:  High 

Problem:  Failure  Modes,  Effects  and  Criticality  Analysis  (FMECA)  is  planned  too  late  to  iden¬ 
tify  and  correct  any  critical  single-point  failure  points  prior  to  design  freeze. 

Action:  Additional  resources  are  being  sought  to  expedite  performance  of  FMECA. 

II.  Risk  Area  Status:  Supportability  Likelihood:  High  Consequence:  Moderate/High 

1.  Title:  Operational  Support  Likelihood:  High  Consequence:  Moderate/High 

Problem:  Power  supply  subcontractor  is  in  financial  trouble  and  may  go  out  of  business.  No 
other  known  sources  exist. 

Action:  Doing  trade  study  to  see  if  alternative  designs  have  a  broader  power  supply  vendor 
base.  Prime  contractor  is  negotiating  with  the  subcontractor  to  buy  drawings  for  develop¬ 
ment  of  second  source. 

Figure  7-1.  Example  Risk  Tracking  Report 
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caught  unprepared  for  a  result.  A  Watch  List  will  be  used  to  track  upcoming  events  and 
activities.  A  sample  Watch  List  is  contained  in  Figure  7-2. 

2.3  PIPT  RISK  SUMMARY  REPORT 

In  addition  to  the  RTRs  for  individual  HIGH  and  MODERATE  risks,  PIPTs  will  prepare  a 
periodic  summary  of  the  ratings  for  all  the  risks  in  their  areas.  Figure  7-3  provides  an 
example  of  this  report.  The  format  for  this  summary  is  based  on  the  Risk  Assessment 
Guide  shown  in  Figure  6-1.  The  entries  in  each  cell  of  the  matrix  represent  the  number  of 
identified  risks  with  the  corresponding  likelihood  and  consequence  values. 


Potential 

Risk  Event 

Risk  Reduction  Actions 

Action  Code 

Due  Date 

Date  Completed 

Explanation 

•  Accurately 

•  Use  multiple  finite 

SE03 

31  Aug  99 

predicting  shock 

element  codes  & 

environment 

simplified  numerical 

shipboard 

models  for  early 

equipment  will 

assessments. 

experience. 

•  Shock  test  simple 

SE03 

31  Aug  99 

isolated  structure, 

simple  isolated  deck, 
and  proposed 
isolated  structure  to 
improve  confidence  in 

predictions. 

•  Evaluating  impact 

•  Concentrate  on 

SE31 

31  Apr  99 

of  circuit  cards 

modeling  and  scale 

that  are  not 

testing  of 

similar  to 

technologies  not 

previous  designs. 

demonstrated 
successfully  in  large- 
scale  tests  or  full- 
scale  tests. 

•  Factor  design  and 

SE032 

31  Aug  99 

into  system 
requirements. 

Continue  model  tests 
to  validate  predictions 
for  isolated  decks. 

Figure  7-2.  Sample  Watch  List 
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Figure  7-3.  Example  PIPT  Risk  Summary  Report 


Design 

Requirements 

Trade 

Studies 

Design 

Process 

Integrated  Test 
Plan 

Failure 

Reporting 

System 

Manufacturing 

Plan 

Development 
of  requirements 
traceability  plan 

Development 
of  specification 
tree 

Specifications 
reviewed  for: 

Definition  of 
all  use 

environments 

Definition  of 
all  functional 
requirements 
for  each 
mission 
performed 

Users  needs 
prioritized 

Alternative 
system  con¬ 
figurations 
selected 

Test  methods 
selected 

Design  require¬ 
ments  stability 

Producibility 

analysis 

conducted 

Design  ana¬ 
lyzed  for: 

Cost 

Parts 

reduction 

Manufac¬ 

turability 

Testability 

Alt  developmen¬ 
tal  tests  at 
system  and 
subsystem  level 
identified 

Identification  of 
who  will  to  test 
(Government, 
contractor, 
supplier)  of 
requirements 
traceability  plan 

Development  of 

specification 

tree 

Specifications 
reviewed  for: 

Definition  of 
all  use  envi¬ 
ronments 

Definition  of 
all  functional 
requirements 
for  each 
mission 
performed 

Contractor 
corporate-level 
management 
involved  in 
failure  reporting 
and  corrective 
action  process 

Responsibility 
for  analysis 
and  corrective 
action  assigned 
to  specific 
individual  with 
close-out  date 

Plan  docu¬ 
ments  methods 
by  which 
design  to  be 
built 

Plan  contains 
sequence  and 
schedule  of 
events  at 
contractor  and 
sub-contractor 
levels  that 
defines  use  of 
materials, 
fabrication  flow, 
test  equipment, 
tools,  facilities, 
and  personnel 

Reflects 
manufacturing 
inclusion  in 
design  pro¬ 
cess.  Includes 
identification 
and  assess¬ 
ment  of  design 
facilities 

Examples  of  Process  Metrics 


Cost 

Schedule 

Cost  variance 

Cost  performance  index 
Estimate  at  completion 
Management  reserve 

Schedule  variance 

Schedule  performance  index 

Design  schedule  performance 
Manufacturing  schedule  performance 
Test  schedule  performance 

Example  of  Cost  and  Schedule  Metrics 
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APPENDIX  C 

GLOSSARY 


ACAT 

Acquisition  Category 

AHP 

Analytical  Hierarchy  Process 

AMSAA 

Army  Materiel  System  Analysis  Activity 

APB 

Acquisition  Program  Baseline 

API/PM 

Acquisition  Program  Integration/ Program  Management 

ASP 

Acquisition  System  Protection 

BCS 

Baseline  Comparison  System 

BIT 

Built-in  Test 

BMP 

Best  Manufacturing  Program 

CAIG 

Cost  Analysis  Improvement  Group 

CAIV 

Cost  As  an  Independent  Variable 

CARDs 

Cost  Analysis  Requirements  Description 

CCA 

Component  Cost  Analysis 

CCDR 

Contractor  Cost  Data  Reporting 

CDF 

Cumulative  Distribution  Function 

CDR 

Critical  Design  Review 

CER 

Cost  Estimating  Relationship 

CPM 

Critical  Path  Method 

CWBS 

Contract  Work  Breakdown  Structure 

DAD 

Defense  Acquisition  Deskbook 

DAU 

Defense  Acquisition  University 

DBMS 

Database  Management  System 

DCMC 

Defense  Contract  Management  Command 

DEM/VAL 

Demonstration/ Validation 

DEARS 

Defense  Federal  Acquisition  Regulation  Supplement 

DoD 

Department  of  Defense 

DoDD 

DoD  Directives 

DPG 

Defense  Planning  Guidance 

C-l 


DSMC 

Defense  Systems  Management  College 

DT&E 

Development,  Test  and  Evaluation 

DTSE&E 

Director,  Test,  Systems  Engineering,  and  Evaluation 

EAC 

Estimate  At  Completion 

EMD 

Engineering  and  Manufacturing  Development 

EMP 

Electromagnetic  Pulse 

ESC 

Electronic  Systems  Center 

ESM 

Electronic  Warfare  Support  Measures 

ESS 

Environmental  Stress  Screening 

EV 

Earned  Value 

FMECA 

Failure  Mode,  Effects  and  Criticality  Analysis 

FRACAS 

Failure,  Reporting,  Analyze,  and  corrective  Action 

GAO 

Government  Accounting  Office 

GFE 

Government  Furnished  Equipment 

HWIL 

Hardware-in-the-Loop 

IBR 

Integrated  Baseline  Review 

IFF 

Identification  Friend  or  Foe 

IIPT 

Integrating  Integrated  Product  Teams 

IMS 

Integrated  Master  Schedule 

IMP 

Integrated  Master  Plan 

IOC 

Initial  Operational  Capability 

IPD 

Integrated  Product  Development 

IPPD 

Integrated  Product  and  Process  Development 

IPT 

Integrated  Product  Teams 

KPP 

Key  Performance  Parameters 

LCC 

Life-Cycle  Cost 

LFT&E 

Live-Fire  Test  and  Evaluation 

LRIP 

Low  Rate  Initial  Production 
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M&E 

Mechanical  and  Electrical 

M&S 

Modeling  and  Simulation 

MAIS 

Major  Automated  Information  System 

MDA 

Milestone  Decision  Authority 

MDAPs 

Major  Defense  Acquisition  Programs 

MIS 

Management  Information  System 

MNS 

Mission  Need  Statement 

MOA 

Memoranda  of  Agreement 

MOU 

Memoranda  of  Understanding 

MS 

Milestone 

MTBF 

Mean  Time  Between  Failure 

NDI 

Non-Developmental  Item 

NSSN 

New  Nuclear  Submarine 

O&M 

Operations  and  Maintenance 

OIPT 

Overarching  Integrated  Product  Team 

OLRDB 

On-Line  Risk  Data  Base 

ORD 

Operational  Requirement  Document 

OSD 

Office  of  the  Secretary  of  Defense 

OT&E 

Operational  Test  and  Evaluation 

PDF 

Probability  Density  Function 

PDRR 

Program  Definition/Risk  Reduction 

PIIPT 

Program  Integrating  Integrated  Product  Team 

PIPT 

Program  Integrated  Product  Team 

PM 

Program  Manager 

PMI 

Project  Management  Institute 

PMO 

Program  Management  Office 

PMWS 

Program  Manager's  Work  Station 

POE 

Program  Office  Estimate 

POM 

Program  Objective  Memorandum 

PRAG 

Performance  Risk  Assessment  Group 

PRR 

Production  Readiness  Review 

PSR 

Program  Status  Report 
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R&D 

Research  and  Development 

'rd&a 

Research,  Development  and  Acquisition 

R&M 

Repairability  and  Maintainability 

RAR 

Risk  Assessment  Report 

RFP 

Request  for  Proposal 

RIF 

Risk  Information  Form 

RMIS 

Risk  Management  Information  System 

RMP 

Risk  Management  Plan 

RTR 

Risk  Tracking  Report 

SEI 

Software  Engineering  Institute 

SME 

Subject-Matter  Expert 

SOW 

Statement  of  Work 

SPMN 

Software  Program  Managers  Network 

SRE 

Software  Risk  Evaluation 

SRR 

System  Requirements  Review 

STA 

Special  Threat  Assessment 

STAR 

Special  Threat  Assessment  Report 

T&E 

Test  and  Evaluation 

TAAF 

Test-Analyze-and-Fix 

TEMP 

Test  and  Evaluation  Master  Plan 

TPM 

Technical  Performance  Measurement 

TRIMS 

Technical  Risk  Identification  and  Mitigation  System 

UAV 

Unmanned  Aerial  Vehicle 

UHF 

Ultra-High  Frequency 

use 

United  States  Code 

USD(A&T) 

Under  Secretary  of  Defense,  Acquisition  and  Technology 

WBS 

Work  Breakdown  Structure 
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